One Weird Trick to Improve Your Semi-Weakly Supervised Semantic Segmentation Model
This work addresses a bottleneck in semantic segmentation for computer vision by offering a simple, incremental improvement that reduces complexity and enhances accuracy.
The paper tackles the problem of semi-weakly supervised semantic segmentation by proposing prediction filtering, a post-processing method that uses classifier confidence to ignore segmentation predictions for absent classes, resulting in competitive or better performance than prior algorithms and further improvements when combined with existing methods.
Semi-weakly supervised semantic segmentation (SWSSS) aims to train a model to identify objects in images based on a small number of images with pixel-level labels, and many more images with only image-level labels. Most existing SWSSS algorithms extract pixel-level pseudo-labels from an image classifier - a very difficult task to do well, hence requiring complicated architectures and extensive hyperparameter tuning on fully-supervised validation sets. We propose a method called prediction filtering, which instead of extracting pseudo-labels, just uses the classifier as a classifier: it ignores any segmentation predictions from classes which the classifier is confident are not present. Adding this simple post-processing method to baselines gives results competitive with or better than prior SWSSS algorithms. Moreover, it is compatible with pseudo-label methods: adding prediction filtering to existing SWSSS algorithms further improves segmentation performance.