Segment Every Out-of-Distribution Object
This addresses safety-critical applications in real-world deployment by improving segmentation accuracy for out-of-distribution objects, representing a strong specific gain rather than a foundational advancement.
The paper tackled the problem of segmenting out-of-distribution objects in semantic segmentation by introducing the S2M method, which directly segments entire objects without threshold selection, resulting in approximately 20% higher IoU and 40% higher mean F1 score compared to state-of-the-art methods across multiple benchmarks.
Semantic segmentation models, while effective for in-distribution categories, face challenges in real-world deployment due to encountering out-of-distribution (OoD) objects. Detecting these OoD objects is crucial for safety-critical applications. Existing methods rely on anomaly scores, but choosing a suitable threshold for generating masks presents difficulties and can lead to fragmentation and inaccuracy. This paper introduces a method to convert anomaly \textbf{S}core \textbf{T}o segmentation \textbf{M}ask, called S2M, a simple and effective framework for OoD detection in semantic segmentation. Unlike assigning anomaly scores to pixels, S2M directly segments the entire OoD object. By transforming anomaly scores into prompts for a promptable segmentation model, S2M eliminates the need for threshold selection. Extensive experiments demonstrate that S2M outperforms the state-of-the-art by approximately 20% in IoU and 40% in mean F1 score, on average, across various benchmarks including Fishyscapes, Segment-Me-If-You-Can, and RoadAnomaly datasets.