Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
This provides a practical method for uncertainty estimation in deep learning, which is crucial for applications like autonomous systems and healthcare, though it is incremental as it builds on existing batch normalization techniques.
The paper tackled the problem of estimating model uncertainty in deep networks by showing that training with batch normalization is equivalent to approximate Bayesian inference, enabling uncertainty estimates without network modifications. The result outperformed baselines with strong statistical significance and achieved competitive performance with recent Bayesian methods.
We show that training a deep network using batch normalization is equivalent to approximate inference in Bayesian models. We further demonstrate that this finding allows us to make meaningful estimates of the model uncertainty using conventional architectures, without modifications to the network or the training procedure. Our approach is thoroughly validated by measuring the quality of uncertainty in a series of empirical experiments on different tasks. It outperforms baselines with strong statistical significance, and displays competitive performance with recent Bayesian approaches.