Exploiting Shape Cues for Weakly Supervised Semantic Segmentation
This work improves weakly supervised semantic segmentation for computer vision applications, offering a more efficient single-stage method with competitive results, though it is incremental as it builds on existing CAM-based pipelines.
The paper tackles the problem of weakly supervised semantic segmentation by addressing the limited coverage of class activation maps, proposing to exploit shape information to improve mask predictions and achieve precise, shape-aligned results, with the model surpassing existing state-of-the-art single-stage approaches by large margins and achieving new state-of-the-art performance in a two-stage pipeline.
Weakly supervised semantic segmentation (WSSS) aims to produce pixel-wise class predictions with only image-level labels for training. To this end, previous methods adopt the common pipeline: they generate pseudo masks from class activation maps (CAMs) and use such masks to supervise segmentation networks. However, it is challenging to derive comprehensive pseudo masks that cover the whole extent of objects due to the local property of CAMs, i.e., they tend to focus solely on small discriminative object parts. In this paper, we associate the locality of CAMs with the texture-biased property of convolutional neural networks (CNNs). Accordingly, we propose to exploit shape information to supplement the texture-biased CNN features, thereby encouraging mask predictions to be not only comprehensive but also well-aligned with object boundaries. We further refine the predictions in an online fashion with a novel refinement method that takes into account both the class and the color affinities, in order to generate reliable pseudo masks to supervise the model. Importantly, our model is end-to-end trained within a single-stage framework and therefore efficient in terms of the training cost. Through extensive experiments on PASCAL VOC 2012, we validate the effectiveness of our method in producing precise and shape-aligned segmentation results. Specifically, our model surpasses the existing state-of-the-art single-stage approaches by large margins. What is more, it also achieves a new state-of-the-art performance over multi-stage approaches, when adopted in a simple two-stage pipeline without bells and whistles.