Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization
This work addresses the problem of improving segmentation accuracy with limited labeled data for computer vision applications, representing an incremental advance in semi-supervised learning techniques.
The paper tackles the challenge of large intra-class variation in semi-supervised semantic segmentation by proposing a prototype-based consistency regularization method to ease label propagation, achieving superior performance over state-of-the-art methods on Pascal VOC and Cityscapes benchmarks.
Semi-supervised semantic segmentation requires the model to effectively propagate the label information from limited annotated images to unlabeled ones. A challenge for such a per-pixel prediction task is the large intra-class variation, i.e., regions belonging to the same class may exhibit a very different appearance even in the same picture. This diversity will make the label propagation hard from pixels to pixels. To address this problem, we propose a novel approach to regularize the distribution of within-class features to ease label propagation difficulty. Specifically, our approach encourages the consistency between the prediction from a linear predictor and the output from a prototype-based predictor, which implicitly encourages features from the same pseudo-class to be close to at least one within-class prototype while staying far from the other between-class prototypes. By further incorporating CutMix operations and a carefully-designed prototype maintenance strategy, we create a semi-supervised semantic segmentation algorithm that demonstrates superior performance over the state-of-the-art methods from extensive experimental evaluation on both Pascal VOC and Cityscapes benchmarks.