Out of Distribution Detection on ImageNet-O
This work provides a comprehensive benchmarking for OOD detection on a novel dataset, aiding research in making machine learning systems more robust, but it is incremental as it focuses on comparative analysis without introducing new methods.
The paper conducted a comparative analysis of out-of-distribution (OOD) detection methods on the ImageNet-O dataset, benchmarking state-of-the-art approaches across various model architectures and settings to evaluate robustness in ImageNet-trained deep neural networks.
Out of distribution (OOD) detection is a crucial part of making machine learning systems robust. The ImageNet-O dataset is an important tool in testing the robustness of ImageNet trained deep neural networks that are widely used across a variety of systems and applications. We aim to perform a comparative analysis of OOD detection methods on ImageNet-O, a first of its kind dataset with a label distribution different than that of ImageNet, that has been created to aid research in OOD detection for ImageNet models. As this dataset is fairly new, we aim to provide a comprehensive benchmarking of some of the current state of the art OOD detection methods on this novel dataset. This benchmarking covers a variety of model architectures, settings where we haves prior access to the OOD data versus when we don't, predictive score based approaches, deep generative approaches to OOD detection, and more.