Simultaneous Semantic Segmentation and Outlier Detection in Presence of Domain Shift
This addresses the challenge of robust real-world road driving applications by enabling simultaneous segmentation and outlier detection, though it is incremental as it builds on existing two-head models and validation methods.
The paper tackles the problem of identifying unreliable image content in semantic segmentation under domain shift by recovering dense outlier maps alongside segmentation in a single forward pass. It achieves state-of-the-art performance on the WildDash benchmark, with validation showing best results from training to discriminate inliers from pasted ImageNet-1k content.
Recent success on realistic road driving datasets has increased interest in exploring robust performance in real-world applications. One of the major unsolved problems is to identify image content which can not be reliably recognized with a given inference engine. We therefore study approaches to recover a dense outlier map alongside the primary task with a single forward pass, by relying on shared convolutional features. We consider semantic segmentation as the primary task and perform extensive validation on WildDash val (inliers), LSUN val (outliers), and pasted objects from Pascal VOC 2007 (outliers). We achieve the best validation performance by training to discriminate inliers from pasted ImageNet-1k content, even though ImageNet-1k contains many road-driving pixels, and, at least nominally, fails to account for the full diversity of the visual world. The proposed two-head model performs comparably to the C-way multi-class model trained to predict uniform distribution in outliers, while outperforming several other validated approaches. We evaluate our best two models on the WildDash test dataset and set a new state of the art on the WildDash benchmark.