Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation
This work addresses mask ambiguity in weakly-supervised semantic segmentation, offering a plug-and-play improvement for existing methods, though it is incremental as it builds directly on CAM variants.
The paper tackles the problem of generating poor pseudo masks in weakly-supervised semantic segmentation by identifying the binary cross-entropy loss in class activation maps as the cause, and introduces ReCAM, which reactivates CAM with softmax cross-entropy loss to reduce mask ambiguity, resulting in high-quality masks on PASCAL VOC and MS COCO datasets.
Extracting class activation maps (CAM) is arguably the most standard step of generating pseudo masks for weakly-supervised semantic segmentation (WSSS). Yet, we find that the crux of the unsatisfactory pseudo masks is the binary cross-entropy loss (BCE) widely used in CAM. Specifically, due to the sum-over-class pooling nature of BCE, each pixel in CAM may be responsive to multiple classes co-occurring in the same receptive field. As a result, given a class, its hot CAM pixels may wrongly invade the area belonging to other classes, or the non-hot ones may be actually a part of the class. To this end, we introduce an embarrassingly simple yet surprisingly effective method: Reactivating the converged CAM with BCE by using softmax cross-entropy loss (SCE), dubbed \textbf{ReCAM}. Given an image, we use CAM to extract the feature pixels of each single class, and use them with the class label to learn another fully-connected layer (after the backbone) with SCE. Once converged, we extract ReCAM in the same way as in CAM. Thanks to the contrastive nature of SCE, the pixel response is disentangled into different classes and hence less mask ambiguity is expected. The evaluation on both PASCAL VOC and MS~COCO shows that ReCAM not only generates high-quality masks, but also supports plug-and-play in any CAM variant with little overhead.