Dual form Complementary Masking for Domain-Adaptive Image Segmentation
This work addresses domain adaptation for image segmentation, offering a new paradigm that enhances feature extraction without separate pre-training, though it is incremental in building on existing masked modeling approaches.
The paper tackled the problem of insufficient theoretical understanding and exploitation of masked image modeling in unsupervised domain adaptation for image segmentation, and proposed MaskTwins, a framework that integrates masked reconstruction to enforce consistency between complementary masks, achieving superior performance over baselines in natural and biological image segmentation.
Recent works have correlated Masked Image Modeling (MIM) with consistency regularization in Unsupervised Domain Adaptation (UDA). However, they merely treat masking as a special form of deformation on the input images and neglect the theoretical analysis, which leads to a superficial understanding of masked reconstruction and insufficient exploitation of its potential in enhancing feature extraction and representation learning. In this paper, we reframe masked reconstruction as a sparse signal reconstruction problem and theoretically prove that the dual form of complementary masks possesses superior capabilities in extracting domain-agnostic image features. Based on this compelling insight, we propose MaskTwins, a simple yet effective UDA framework that integrates masked reconstruction directly into the main training pipeline. MaskTwins uncovers intrinsic structural patterns that persist across disparate domains by enforcing consistency between predictions of images masked in complementary ways, enabling domain generalization in an end-to-end manner. Extensive experiments verify the superiority of MaskTwins over baseline methods in natural and biological image segmentation. These results demonstrate the significant advantages of MaskTwins in extracting domain-invariant features without the need for separate pre-training, offering a new paradigm for domain-adaptive segmentation.