On Advantages of Mask-level Recognition for Outlier-aware Segmentation
This work addresses the challenge of robust segmentation in real-world applications where outliers are common, offering incremental improvements over existing methods.
The paper tackles the problem of outlier-aware semantic segmentation by showing that mask-level predictions improve performance in the presence of outliers, even without fine-tuning, and proposes a new uncertainty formulation that reduces false positives at borders, achieving state-of-the-art results with and without negative data training.
Most dense recognition approaches bring a separate decision in each particular pixel. These approaches deliver competitive performance in usual closed-set setups. However, important applications in the wild typically require strong performance in presence of outliers. We show that this demanding setup greatly benefit from mask-level predictions, even in the case of non-finetuned baseline models. Moreover, we propose an alternative formulation of dense recognition uncertainty that effectively reduces false positive responses at semantic borders. The proposed formulation produces a further improvement over a very strong baseline and sets the new state of the art in outlier-aware semantic segmentation with and without training on negative data. Our contributions also lead to performance improvement in a recent panoptic setup. In-depth experiments confirm that our approach succeeds due to implicit aggregation of pixel-level cues into mask-level predictions.