Three Factors to Improve Out-of-Distribution Detection
This work addresses a specific problem in OOD detection for machine learning practitioners, offering incremental improvements to existing methods.
The paper tackles the trade-off between classification accuracy and out-of-distribution (OOD) detection performance by incorporating three factors: self-knowledge distillation loss, semi-hard outlier data sampling, and supervised contrastive learning, resulting in improvements in both accuracy and OOD detection metrics like AUROC, FPR, and AUPR.
In the problem of out-of-distribution (OOD) detection, the usage of auxiliary data as outlier data for fine-tuning has demonstrated encouraging performance. However, previous methods have suffered from a trade-off between classification accuracy (ACC) and OOD detection performance (AUROC, FPR, AUPR). To improve this trade-off, we make three contributions: (i) Incorporating a self-knowledge distillation loss can enhance the accuracy of the network; (ii) Sampling semi-hard outlier data for training can improve OOD detection performance with minimal impact on accuracy; (iii) The introduction of our novel supervised contrastive learning can simultaneously improve OOD detection performance and the accuracy of the network. By incorporating all three factors, our approach enhances both accuracy and OOD detection performance by addressing the trade-off between classification and OOD detection. Our method achieves improvements over previous approaches in both performance metrics.