Detecting Out-of-Distribution Through the Lens of Neural Collapse
This addresses the challenge of safe AI deployment by improving OOD detection generalization and efficiency, though it appears incremental as it builds on existing Neural Collapse insights.
The paper tackled the problem of out-of-distribution (OOD) detection by proposing a method inspired by Neural Collapse, which leverages feature proximity to weight vectors and feature norms to filter OOD samples, resulting in robust performance across diverse scenarios with inference latency comparable to basic softmax-confidence detectors.
Out-of-Distribution (OOD) detection is critical for safe deployment; however, existing detectors often struggle to generalize across datasets of varying scales and model architectures, and some can incur high computational costs in real-world applications. Inspired by the phenomenon of Neural Collapse, we propose a versatile and efficient OOD detection method. Specifically, we re-characterize prior observations that in-distribution (ID) samples form clusters, demonstrating that, with appropriate centering, these clusters align closely with model weight vectors. Additionally, we reveal that ID features tend to expand into a simplex Equiangular Tight Frame, explaining the common observation that ID features are situated farther from the origin than OOD features. Incorporating both insights from Neural Collapse, our OOD detector leverages feature proximity to weight vectors and complements this approach by using feature norms to effectively filter out OOD samples. Extensive experiments on off-the-shelf models demonstrate the robustness of our OOD detector across diverse scenarios, mitigating generalization discrepancies and enhancing overall performance, with inference latency comparable to that of the basic softmax-confidence detector. Code is available here: https://github.com/litianliu/NCI-OOD.