SemFormer: Semantic Guided Activation Transformer for Weakly Supervised Semantic Segmentation
This work addresses the problem of semantic segmentation with weak supervision for computer vision researchers, offering a novel transformer-based method that improves performance over existing CNN-based approaches.
The paper tackles weakly supervised semantic segmentation by proposing SemFormer, a transformer-based framework that uses class embeddings and learned semantics to guide activation map generation, achieving 74.3% mIoU on PASCAL VOC 2012 and surpassing many recent approaches.
Recent mainstream weakly supervised semantic segmentation (WSSS) approaches are mainly based on Class Activation Map (CAM) generated by a CNN (Convolutional Neural Network) based image classifier. In this paper, we propose a novel transformer-based framework, named Semantic Guided Activation Transformer (SemFormer), for WSSS. We design a transformer-based Class-Aware AutoEncoder (CAAE) to extract the class embeddings for the input image and learn class semantics for all classes of the dataset. The class embeddings and learned class semantics are then used to guide the generation of activation maps with four losses, i.e., class-foreground, class-background, activation suppression, and activation complementation loss. Experimental results show that our SemFormer achieves \textbf{74.3}\% mIoU and surpasses many recent mainstream WSSS approaches by a large margin on PASCAL VOC 2012 dataset. Code will be available at \url{https://github.com/JLChen-C/SemFormer}.