A statistical framework for efficient out of distribution detection in deep neural networks
This addresses the critical issue of DNN reliability in real-world applications like autonomous vehicles and healthcare, though it is an incremental improvement over existing OOD detection methods.
The authors tackled the problem of unreliable deep neural network predictions on out-of-distribution (OOD) data by framing OOD detection as a statistical hypothesis testing problem, achieving comparable or better results than state-of-the-art methods on benchmarks without retraining or prior knowledge, and at reduced computational cost.
Background. Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn from a distribution similar to that of the training set. However, DNNs' predictions are brittle and unreliable when the test samples are drawn from a dissimilar distribution. This is a major concern for deployment in real-world applications, where such behavior may come at a considerable cost, such as industrial production lines, autonomous vehicles, or healthcare applications. Contributions. We frame Out Of Distribution (OOD) detection in DNNs as a statistical hypothesis testing problem. Tests generated within our proposed framework combine evidence from the entire network. Unlike previous OOD detection heuristics, this framework returns a $p$-value for each test sample. It is guaranteed to maintain the Type I Error (T1E - incorrectly predicting OOD for an actual in-distribution sample) for test data. Moreover, this allows to combine several detectors while maintaining the T1E. Building on this framework, we suggest a novel OOD procedure based on low-order statistics. Our method achieves comparable or better results than state-of-the-art methods on well-accepted OOD benchmarks, without retraining the network parameters or assuming prior knowledge on the test distribution -- and at a fraction of the computational cost.