MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation
This work addresses domain adaptation challenges in semantic segmentation, particularly for applications like autonomous driving, by improving accuracy on ambiguous objects and fine details, though it is incremental as it builds on existing UDA methods.
The paper tackles the problem of inferior performance on fine structures and oversegmentation in unsupervised domain adaptation for semantic segmentation by leveraging geometric depth information. It introduces MICDrop, a method using complementary masking of image and depth features, achieving new state-of-the-art results across standard benchmarks.
Unsupervised Domain Adaptation (UDA) is the task of bridging the domain gap between a labeled source domain, e.g., synthetic data, and an unlabeled target domain. We observe that current UDA methods show inferior results on fine structures and tend to oversegment objects with ambiguous appearance. To address these shortcomings, we propose to leverage geometric information, i.e., depth predictions, as depth discontinuities often coincide with segmentation boundaries. We show that naively incorporating depth into current UDA methods does not fully exploit the potential of this complementary information. To this end, we present MICDrop, which learns a joint feature representation by masking image encoder features while inversely masking depth encoder features. With this simple yet effective complementary masking strategy, we enforce the use of both modalities when learning the joint feature representation. To aid this process, we propose a feature fusion module to improve both global as well as local information sharing while being robust to errors in the depth predictions. We show that our method can be plugged into various recent UDA methods and consistently improve results across standard UDA benchmarks, obtaining new state-of-the-art performances.