Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement
This work addresses the critical need for reliable OOD detection in deep learning systems, offering incremental improvements over existing methods.
The paper tackled the problem of out-of-distribution (OOD) detection in deep learning by proposing SCALE, a post-hoc method that enhances OOD detection without harming in-distribution accuracy, and ISH, a training-time method, achieving AUROC improvements of +1.85% for near-OOD and +0.74% for far-OOD on the OpenOOD v1.5 ImageNet-1K benchmark.
The capacity of a modern deep learning system to determine if a sample falls within its realm of knowledge is fundamental and important. In this paper, we offer insights and analyses of recent state-of-the-art out-of-distribution (OOD) detection methods - extremely simple activation shaping (ASH). We demonstrate that activation pruning has a detrimental effect on OOD detection, while activation scaling enhances it. Moreover, we propose SCALE, a simple yet effective post-hoc network enhancement method for OOD detection, which attains state-of-the-art OOD detection performance without compromising in-distribution (ID) accuracy. By integrating scaling concepts into the training process to capture a sample's ID characteristics, we propose Intermediate Tensor SHaping (ISH), a lightweight method for training time OOD detection enhancement. We achieve AUROC scores of +1.85\% for near-OOD and +0.74\% for far-OOD datasets on the OpenOOD v1.5 ImageNet-1K benchmark. Our code and models are available at https://github.com/kai422/SCALE.