OpenMix: Exploring Outlier Samples for Misclassification Detection
This work addresses the critical need for reliable confidence estimation in high-stakes applications, offering a unified framework for detecting both misclassified and out-of-distribution samples.
The paper tackles the problem of overconfident misclassifications in deep neural networks by proposing OpenMix, a method that uses outlier samples to improve misclassification detection, achieving significant improvements in confidence reliability across various scenarios.
Reliable confidence estimation for deep neural classifiers is a challenging yet fundamental requirement in high-stakes applications. Unfortunately, modern deep neural networks are often overconfident for their erroneous predictions. In this work, we exploit the easily available outlier samples, i.e., unlabeled samples coming from non-target classes, for helping detect misclassification errors. Particularly, we find that the well-known Outlier Exposure, which is powerful in detecting out-of-distribution (OOD) samples from unknown classes, does not provide any gain in identifying misclassification errors. Based on these observations, we propose a novel method called OpenMix, which incorporates open-world knowledge by learning to reject uncertain pseudo-samples generated via outlier transformation. OpenMix significantly improves confidence reliability under various scenarios, establishing a strong and unified framework for detecting both misclassified samples from known classes and OOD samples from unknown classes. The code is publicly available at https://github.com/Impression2805/OpenMix.