Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detection
This addresses the problem of detecting anomalies in dense, structured data like driving scenes for applications such as autonomous vehicles, though it is incremental as it extends existing nearest-neighbor methods to a new domain.
The paper tackled dense out-of-distribution detection in complex driving scenes by using nearest-neighbor approaches with transformer-based feature representations, achieving state-of-the-art results on benchmarks like RoadAnomaly, StreetHazards, and SegmentMeIfYouCan-Anomaly.
The key to out-of-distribution detection is density estimation of the in-distribution data or of its feature representations. This is particularly challenging for dense anomaly detection in domains where the in-distribution data has a complex underlying structure. Nearest-Neighbors approaches have been shown to work well in object-centric data domains, such as industrial inspection and image classification. In this paper, we show that nearest-neighbor approaches also yield state-of-the-art results on dense novelty detection in complex driving scenes when working with an appropriate feature representation. In particular, we find that transformer-based architectures produce representations that yield much better similarity metrics for the task. We identify the multi-head structure of these models as one of the reasons, and demonstrate a way to transfer some of the improvements to CNNs. Ultimately, the approach is simple and non-invasive, i.e., it does not affect the primary segmentation performance, refrains from training on examples of anomalies, and achieves state-of-the-art results on RoadAnomaly, StreetHazards, and SegmentMeIfYouCan-Anomaly.