Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks
This addresses a safety issue in machine learning by exploring OOD detection without requiring additional OOD samples or model modifications, though it appears incremental as it builds on existing soft labeling methods.
The study investigated how soft labeling, a common output regularization technique, affects out-of-distribution (OOD) detection in deep neural networks, finding that it can either deteriorate or improve OOD detection performance based on how incorrect class outputs are regularized.
Soft labeling becomes a common output regularization for generalization and model compression of deep neural networks. However, the effect of soft labeling on out-of-distribution (OOD) detection, which is an important topic of machine learning safety, is not explored. In this study, we show that soft labeling can determine OOD detection performance. Specifically, how to regularize outputs of incorrect classes by soft labeling can deteriorate or improve OOD detection. Based on the empirical results, we postulate a future work for OOD-robust DNNs: a proper output regularization by soft labeling can construct OOD-robust DNNs without additional training of OOD samples or modifying the models, while improving classification accuracy.