Synthetic to Real Adaptation with Generative Correlation Alignment Networks
This addresses the challenge of poor model performance when trained on synthetic data for real-world applications, offering a domain adaptation solution that is incremental over existing methods.
The paper tackles the problem of domain discrepancy between synthetic and real images for object recognition by proposing a Deep Generative Correlation Alignment Network (DGCAN) that synthesizes images to minimize this gap, resulting in significant performance boosts on real image benchmarks like PASCAL VOC 2007 and Office dataset.
Synthetic images rendered from 3D CAD models are useful for augmenting training data for object recognition algorithms. However, the generated images are non-photorealistic and do not match real image statistics. This leads to a large domain discrepancy, causing models trained on synthetic data to perform poorly on real domains. Recent work has shown the great potential of deep convolutional neural networks to generate realistic images, but has not utilized generative models to address synthetic-to-real domain adaptation. In this work, we propose a Deep Generative Correlation Alignment Network (DGCAN) to synthesize images using a novel domain adaption algorithm. DGCAN leverages a shape preserving loss and a low level statistic matching loss to minimize the domain discrepancy between synthetic and real images in deep feature space. Experimentally, we show training off-the-shelf classifiers on the newly generated data can significantly boost performance when testing on the real image domains (PASCAL VOC 2007 benchmark and Office dataset), improving upon several existing methods.