Multiple Fusion Adaptation: A Strong Framework for Unsupervised Semantic Segmentation Adaptation
This work addresses the problem of improving segmentation accuracy across domains without extra annotation for computer vision applications, representing an incremental advance in pseudo-label-based adaptation.
The paper tackles unsupervised domain adaptation for semantic segmentation by proposing the Multiple Fusion Adaptation (MFA) method, which achieves state-of-the-art results of 58.2% and 62.5% mIoU on GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes benchmarks.
This paper challenges the cross-domain semantic segmentation task, aiming to improve the segmentation accuracy on the unlabeled target domain without incurring additional annotation. Using the pseudo-label-based unsupervised domain adaptation (UDA) pipeline, we propose a novel and effective Multiple Fusion Adaptation (MFA) method. MFA basically considers three parallel information fusion strategies, i.e., the cross-model fusion, temporal fusion and a novel online-offline pseudo label fusion. Specifically, the online-offline pseudo label fusion encourages the adaptive training to pay additional attention to difficult regions that are easily ignored by offline pseudo labels, therefore retaining more informative details. While the other two fusion strategies may look standard, MFA pays significant efforts to raise the efficiency and effectiveness for integration, and succeeds in injecting all the three strategies into a unified framework. Experiments on two widely used benchmarks, i.e., GTA5-to-Cityscapes and SYNTHIA-to-Cityscapes, show that our method significantly improves the semantic segmentation adaptation, and sets up new state of the art (58.2% and 62.5% mIoU, respectively). The code will be available at https://github.com/KaiiZhang/MFA.