SPIGAN: Privileged Adversarial Learning from Simulation
This addresses the problem of reducing domain shift for researchers and practitioners using synthetic data in computer vision, though it is incremental as it builds on existing unsupervised domain adaptation techniques.
The paper tackles the domain gap between synthetic and real-world data in computer vision by proposing SPIGAN, an unsupervised domain adaptation algorithm that uses simulator privileged information and GANs, resulting in improved performance over no adaptation and state-of-the-art methods on semantic segmentation tasks with Cityscapes and Vistas datasets.
Deep Learning for Computer Vision depends mainly on the source of supervision.Photo-realistic simulators can generate large-scale automatically labeled syntheticdata, but introduce a domain gap negatively impacting performance. We propose anew unsupervised domain adaptation algorithm, called SPIGAN, relying on Sim-ulator Privileged Information (PI) and Generative Adversarial Networks (GAN).We use internal data from the simulator as PI during the training of a target tasknetwork. We experimentally evaluate our approach on semantic segmentation. Wetrain the networks on real-world Cityscapes and Vistas datasets, using only unla-beled real-world images and synthetic labeled data with z-buffer (depth) PI fromthe SYNTHIA dataset. Our method improves over no adaptation and state-of-the-art unsupervised domain adaptation techniques.