An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problems
This work addresses the problem of reducing annotation costs for semantic segmentation by exploring SAM's potential in WSSS, representing an incremental application of an existing model to a new task.
The researchers applied the Segment Anything Model (SAM) to weakly-supervised semantic segmentation (WSSS) by using it as a pseudo-label generator from image-level labels, achieving remarkable improvements over state-of-the-art methods on PASCAL VOC and MS-COCO datasets.
The Segment Anything Model (SAM) has demonstrated exceptional performance and versatility, making it a promising tool for various related tasks. In this report, we explore the application of SAM in Weakly-Supervised Semantic Segmentation (WSSS). Particularly, we adapt SAM as the pseudo-label generation pipeline given only the image-level class labels. While we observed impressive results in most cases, we also identify certain limitations. Our study includes performance evaluations on PASCAL VOC and MS-COCO, where we achieved remarkable improvements over the latest state-of-the-art methods on both datasets. We anticipate that this report encourages further explorations of adopting SAM in WSSS, as well as wider real-world applications.