Supervision Adaptation Balancing In-distribution Generalization and Out-of-distribution Detection
This addresses a critical challenge in non-IID deep learning for improving OOD-tolerant ID classification, though it appears incremental as it builds on existing methods that add OOD samples to training.
The paper tackles the problem of distributional vulnerability in deep neural networks, where models make high-confidence predictions for out-of-distribution (OOD) samples, by introducing a supervision adaptation approach that generates adaptive supervision for OOD samples to balance in-distribution (ID) generalization and OOD detection, with experiments showing improved performance on ID classification and OOD detection across multiple datasets.
The discrepancy between in-distribution (ID) and out-of-distribution (OOD) samples can lead to \textit{distributional vulnerability} in deep neural networks, which can subsequently lead to high-confidence predictions for OOD samples. This is mainly due to the absence of OOD samples during training, which fails to constrain the network properly. To tackle this issue, several state-of-the-art methods include adding extra OOD samples to training and assign them with manually-defined labels. However, this practice can introduce unreliable labeling, negatively affecting ID classification. The distributional vulnerability presents a critical challenge for non-IID deep learning, which aims for OOD-tolerant ID classification by balancing ID generalization and OOD detection. In this paper, we introduce a novel \textit{supervision adaptation} approach to generate adaptive supervision information for OOD samples, making them more compatible with ID samples. Firstly, we measure the dependency between ID samples and their labels using mutual information, revealing that the supervision information can be represented in terms of negative probabilities across all classes. Secondly, we investigate data correlations between ID and OOD samples by solving a series of binary regression problems, with the goal of refining the supervision information for more distinctly separable ID classes. Our extensive experiments on four advanced network architectures, two ID datasets, and eleven diversified OOD datasets demonstrate the efficacy of our supervision adaptation approach in improving both ID classification and OOD detection capabilities.