Multi-View Correlation Consistency for Semi-Supervised Semantic Segmentation
This work addresses the challenge of limited labeled data in semantic segmentation for computer vision applications, presenting an incremental improvement over existing methods.
The paper tackles the problem of semi-supervised semantic segmentation by proposing multi-view correlation consistency learning, which combines consistency and contrastive learning to leverage pairwise relationships, achieving 76.8% mIoU on Cityscapes with 1/8 labeled data, just 0.6% below the fully supervised baseline.
Semi-supervised semantic segmentation needs rich and robust supervision on unlabeled data. Consistency learning enforces the same pixel to have similar features in different augmented views, which is a robust signal but neglects relationships with other pixels. In comparison, contrastive learning considers rich pairwise relationships, but it can be a conundrum to assign binary positive-negative supervision signals for pixel pairs. In this paper, we take the best of both worlds and propose multi-view correlation consistency (MVCC) learning: it considers rich pairwise relationships in self-correlation matrices and matches them across views to provide robust supervision. Together with this correlation consistency loss, we propose a view-coherent data augmentation strategy that guarantees pixel-pixel correspondence between different views. In a series of semi-supervised settings on two datasets, we report competitive accuracy compared with the state-of-the-art methods. Notably, on Cityscapes, we achieve 76.8% mIoU with 1/8 labeled data, just 0.6% shy from the fully supervised oracle.