A Mixture of Exemplars Approach for Efficient Out-of-Distribution Detection with Foundation Models
This addresses the challenge of efficient OOD detection for users of deep neural networks in image classification, offering a more practical solution with lower computational costs, though it is incremental as it builds on existing foundation model approaches.
The paper tackles the problem of efficient out-of-distribution (OOD) detection in image classification by proposing Mixture of Exemplars (MoLAR), which uses a frozen pretrained foundation model backbone and compares OOD examples to a small set of exemplars, achieving strong OOD detection performance with reduced inference overhead compared to methods requiring the full in-distribution dataset.
One of the early weaknesses identified in deep neural networks trained for image classification tasks was their inability to provide low confidence predictions on out-of-distribution (OOD) data that was significantly different from the in-distribution (ID) data used to train them. Representation learning, where neural networks are trained in specific ways that improve their ability to detect OOD examples, has emerged as a promising solution. However, these approaches require long training times and can add additional overhead to detect OOD examples. Recent developments in Vision Transformer (ViT) foundation models$\unicode{x2013}$large networks trained on large and diverse datasets with self-supervised approaches$\unicode{x2013}$also show strong performance in OOD detection, and could address these challenges. This paper presents Mixture of Exemplars (MoLAR), an efficient approach to tackling OOD detection challenges that is designed to maximise the benefit of training a classifier with a high quality, frozen, pretrained foundation model backbone. MoLAR provides strong OOD detection performance when only comparing the similarity of OOD examples to the exemplars, a small set of images chosen to be representative of the dataset, leading to significantly reduced overhead for OOD detection inference over other methods that provide best performance when the full ID dataset is used. Extensive experiments demonstrate the improved OOD detection performance of MoLAR in comparison to comparable approaches in both supervised and semi-supervised settings, and code is available at github.com/emannix/molar-mixture-of-exemplars.