Unsupervised Domain Adaptation with Multiple Domain Discriminators and Adaptive Self-Training
This work addresses the domain shift problem in semantic segmentation for urban scenes, offering an incremental improvement over existing methods by combining adversarial learning and self-training.
The paper tackles unsupervised domain adaptation for semantic segmentation of urban scenes by proposing a framework with multiple domain discriminators and adaptive self-training, achieving improved performance on real-world datasets like Cityscapes and Mapillary when adapting from synthetic data such as GTA5 and SYNTHIA.
Unsupervised Domain Adaptation (UDA) aims at improving the generalization capability of a model trained on a source domain to perform well on a target domain for which no labeled data is available. In this paper, we consider the semantic segmentation of urban scenes and we propose an approach to adapt a deep neural network trained on synthetic data to real scenes addressing the domain shift between the two different data distributions. We introduce a novel UDA framework where a standard supervised loss on labeled synthetic data is supported by an adversarial module and a self-training strategy aiming at aligning the two domain distributions. The adversarial module is driven by a couple of fully convolutional discriminators dealing with different domains: the first discriminates between ground truth and generated maps, while the second between segmentation maps coming from synthetic or real world data. The self-training module exploits the confidence estimated by the discriminators on unlabeled data to select the regions used to reinforce the learning process. Furthermore, the confidence is thresholded with an adaptive mechanism based on the per-class overall confidence. Experimental results prove the effectiveness of the proposed strategy in adapting a segmentation network trained on synthetic datasets like GTA5 and SYNTHIA, to real world datasets like Cityscapes and Mapillary.