Towards In-distribution Compatibility in Out-of-distribution Detection
This addresses the issue of reliable anomaly detection in deep learning for applications like safety-critical systems, though it appears incremental as it builds on existing outlier training approaches.
The paper tackles the problem of deep neural networks performing poorly in detecting anomalous out-of-distribution data, which can be worsened by existing outlier training methods that harm in-distribution learning. It proposes a new method that achieves state-of-the-art out-of-distribution detection performance and improves in-distribution accuracy on several benchmarks.
Deep neural network, despite its remarkable capability of discriminating targeted in-distribution samples, shows poor performance on detecting anomalous out-of-distribution data. To address this defect, state-of-the-art solutions choose to train deep networks on an auxiliary dataset of outliers. Various training criteria for these auxiliary outliers are proposed based on heuristic intuitions. However, we find that these intuitively designed outlier training criteria can hurt in-distribution learning and eventually lead to inferior performance. To this end, we identify three causes of the in-distribution incompatibility: contradictory gradient, false likelihood, and distribution shift. Based on our new understandings, we propose a new out-of-distribution detection method by adapting both the top-design of deep models and the loss function. Our method achieves in-distribution compatibility by pursuing less interference with the probabilistic characteristic of in-distribution features. On several benchmarks, our method not only achieves the state-of-the-art out-of-distribution detection performance but also improves the in-distribution accuracy.