Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation
This work addresses the challenge of limited data augmentations in semi-supervised segmentation for computer vision applications, offering a novel approach that is incremental but with strong performance gains.
The paper tackled the problem of improving semi-supervised semantic segmentation by revisiting the weak-to-strong consistency framework, proposing UniMatch with dual-stream perturbations to expand the perturbation space, which significantly surpassed all existing methods on benchmarks like Pascal, Cityscapes, and COCO.
In this work, we revisit the weak-to-strong consistency framework, popularized by FixMatch from semi-supervised classification, where the prediction of a weakly perturbed image serves as supervision for its strongly perturbed version. Intriguingly, we observe that such a simple pipeline already achieves competitive results against recent advanced works, when transferred to our segmentation scenario. Its success heavily relies on the manual design of strong data augmentations, however, which may be limited and inadequate to explore a broader perturbation space. Motivated by this, we propose an auxiliary feature perturbation stream as a supplement, leading to an expanded perturbation space. On the other, to sufficiently probe original image-level augmentations, we present a dual-stream perturbation technique, enabling two strong views to be simultaneously guided by a common weak view. Consequently, our overall Unified Dual-Stream Perturbations approach (UniMatch) surpasses all existing methods significantly across all evaluation protocols on the Pascal, Cityscapes, and COCO benchmarks. Its superiority is also demonstrated in remote sensing interpretation and medical image analysis. We hope our reproduced FixMatch and our results can inspire more future works. Code and logs are available at https://github.com/LiheYoung/UniMatch.