Deep Residual Flow for Out of Distribution Detection
This addresses the challenge of reliable neural network deployment in real-world scenarios by enhancing out-of-distribution detection, though it is incremental as it builds upon existing state-of-the-art methods.
The paper tackled the problem of detecting out-of-distribution examples in neural networks by introducing a novel normalizing flow-based method, improving the true negative rate from 56.7% to 77.5% at a 95% true positive rate on a ResNet trained on CIFAR-100 and evaluated on ImageNet.
The effective application of neural networks in the real-world relies on proficiently detecting out-of-distribution examples. Contemporary methods seek to model the distribution of feature activations in the training data for adequately distinguishing abnormalities, and the state-of-the-art method uses Gaussian distribution models. In this work, we present a novel approach that improves upon the state-of-the-art by leveraging an expressive density model based on normalizing flows. We introduce the residual flow, a novel flow architecture that learns the residual distribution from a base Gaussian distribution. Our model is general, and can be applied to any data that is approximately Gaussian. For out of distribution detection in image datasets, our approach provides a principled improvement over the state-of-the-art. Specifically, we demonstrate the effectiveness of our method in ResNet and DenseNet architectures trained on various image datasets. For example, on a ResNet trained on CIFAR-100 and evaluated on detection of out-of-distribution samples from the ImageNet dataset, holding the true positive rate (TPR) at $95\%$, we improve the true negative rate (TNR) from $56.7\%$ (current state-of-the-art) to $77.5\%$ (ours).