Geometric Unsupervised Domain Adaptation for Semantic Segmentation
This addresses the problem of leveraging synthetic data for real-world semantic segmentation tasks, which is incremental as it builds on existing UDA methods with a novel geometric approach.
The paper tackles the domain gap between synthetic and real data in unsupervised domain adaptation for semantic segmentation by using self-supervised monocular depth estimation as a proxy task, achieving state-of-the-art results on three benchmarks.
Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation. However, they introduce a domain gap that severely hurts real-world performance. We propose to use self-supervised monocular depth estimation as a proxy task to bridge this gap and improve sim-to-real unsupervised domain adaptation (UDA). Our Geometric Unsupervised Domain Adaptation method (GUDA) learns a domain-invariant representation via a multi-task objective combining synthetic semantic supervision with real-world geometric constraints on videos. GUDA establishes a new state of the art in UDA for semantic segmentation on three benchmarks, outperforming methods that use domain adversarial learning, self-training, or other self-supervised proxy tasks. Furthermore, we show that our method scales well with the quality and quantity of synthetic data while also improving depth prediction.