Uncertainty Calibration in Bayesian Neural Networks via Distance-Aware Priors
This addresses the issue of unreliable uncertainty estimates in Bayesian deep learning for applications requiring robust out-of-domain predictions, though it is incremental as it builds on existing calibration methods.
The paper tackled the problem of overconfidence in Bayesian neural networks when making predictions outside the training domain, and introduced Distance-Aware Prior calibration, which improved uncertainty calibration in classification and regression tasks, as demonstrated on benchmarks designed to test predictive distributions away from the data.
As we move away from the data, the predictive uncertainty should increase, since a great variety of explanations are consistent with the little available information. We introduce Distance-Aware Prior (DAP) calibration, a method to correct overconfidence of Bayesian deep learning models outside of the training domain. We define DAPs as prior distributions over the model parameters that depend on the inputs through a measure of their distance from the training set. DAP calibration is agnostic to the posterior inference method, and it can be performed as a post-processing step. We demonstrate its effectiveness against several baselines in a variety of classification and regression problems, including benchmarks designed to test the quality of predictive distributions away from the data.