Domain Adaptation of Synthetic Driving Datasets for Real-World Autonomous Driving
This work addresses the problem of performance degradation in autonomous driving perception models when trained on synthetic data, offering an incremental improvement for domain adaptation in this domain.
The paper tackles the domain gap between synthetic and real-world driving datasets by improving domain adaptation methods, specifically enhancing UNIT-GAN with semantic supervision in pair selection, which boosts performance and visual quality on Cityscapes and Synscapes datasets.
While developing perception based deep learning models, the benefit of synthetic data is enormous. However, performance of networks trained with synthetic data for certain computer vision tasks degrade significantly when tested on real world data due to the domain gap between them. One of the popular solutions in bridging this gap between synthetic and actual world data is to frame it as a domain adaptation task. In this paper, we propose and evaluate novel ways for the betterment of such approaches. In particular we build upon the method of UNIT-GAN. In normal GAN training for the task of domain translation, pairing of images from both the domains (viz, real and synthetic) is done randomly. We propose a novel method to efficiently incorporate semantic supervision into this pair selection, which helps in boosting the performance of the model along with improving the visual quality of such transformed images. We illustrate our empirical findings on Cityscapes \cite{cityscapes} and challenging synthetic dataset Synscapes. Though the findings are reported on the base network of UNIT-GAN, they can be easily extended to any other similar network.