Unsupervised Out-of-Distribution Detection with Batch Normalization
This addresses a critical issue for machine learning practitioners in ensuring model reliability by improving OoD detection, though it appears incremental as it builds on existing batch normalization techniques.
The paper tackles the problem of out-of-distribution detection in generative models, where traditional likelihood-based methods fail by assigning higher likelihood to OoD samples, and proposes a method using batch normalization to exploit in-batch dependencies, resulting in more robust detection for high-dimensional images.
Likelihood from a generative model is a natural statistic for detecting out-of-distribution (OoD) samples. However, generative models have been shown to assign higher likelihood to OoD samples compared to ones from the training distribution, preventing simple threshold-based detection rules. We demonstrate that OoD detection fails even when using more sophisticated statistics based on the likelihoods of individual samples. To address these issues, we propose a new method that leverages batch normalization. We argue that batch normalization for generative models challenges the traditional i.i.d. data assumption and changes the corresponding maximum likelihood objective. Based on this insight, we propose to exploit in-batch dependencies for OoD detection. Empirical results suggest that this leads to more robust detection for high-dimensional images.