Distilling the Unknown to Unveil Certainty
This addresses the problem of network robustness and reliability for AI systems by improving OOD detection, though it appears incremental as it builds on existing knowledge distillation and OOD methods.
The paper tackles out-of-distribution (OOD) detection by proposing a flexible framework that uses confidence amendment to transform OOD samples into in-distribution ones and adjust prediction confidence, enabling training of a binary classifier for OOD detection with and without access to training data, achieving efficacy across various datasets and architectures.
Out-of-distribution (OOD) detection is critical for identifying test samples that deviate from in-distribution (ID) data, ensuring network robustness and reliability. This paper presents a flexible framework for OOD knowledge distillation that extracts OOD-sensitive information from a network to develop a binary classifier capable of distinguishing between ID and OOD samples in both scenarios, with and without access to training ID data. To accomplish this, we introduce Confidence Amendment (CA), an innovative methodology that transforms an OOD sample into an ID one while progressively amending prediction confidence derived from the network to enhance OOD sensitivity. This approach enables the simultaneous synthesis of both ID and OOD samples, each accompanied by an adjusted prediction confidence, thereby facilitating the training of a binary classifier sensitive to OOD. Theoretical analysis provides bounds on the generalization error of the binary classifier, demonstrating the pivotal role of confidence amendment in enhancing OOD sensitivity. Extensive experiments spanning various datasets and network architectures confirm the efficacy of the proposed method in detecting OOD samples.