How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection?
This addresses a key limitation in deploying machine learning models safely by improving OOD detection for high-dimensional data, though it is an incremental advance over existing distance-based methods.
The paper tackles the curse-of-dimensionality problem in out-of-distribution detection by proposing the Subspace Nearest Neighbor framework, which reduces the average FPR95 by 15.96% on CIFAR-100 compared to the best existing method.
Machine learning models deployed in the wild can be challenged by out-of-distribution (OOD) data from unknown classes. Recent advances in OOD detection rely on distance measures to distinguish samples that are relatively far away from the in-distribution (ID) data. Despite the promise, distance-based methods can suffer from the curse-of-dimensionality problem, which limits the efficacy in high-dimensional feature space. To combat this problem, we propose a novel framework, Subspace Nearest Neighbor (SNN), for OOD detection. In training, our method regularizes the model and its feature representation by leveraging the most relevant subset of dimensions (i.e. subspace). Subspace learning yields highly distinguishable distance measures between ID and OOD data. We provide comprehensive experiments and ablations to validate the efficacy of SNN. Compared to the current best distance-based method, SNN reduces the average FPR95 by 15.96% on the CIFAR-100 benchmark.