Reliable Uncertainties for Bayesian Neural Networks using Alpha-divergences
This addresses the issue of unreliable uncertainty estimates in BNNs for practitioners in machine learning, but it is incremental as it builds on existing calibration methods using alpha-divergences.
The paper tackled the problem of uncalibrated uncertainties in Bayesian Neural Networks (BNNs), which often lead to overconfidence, by proposing calibration methods based on alpha-divergences from Information Geometry. The result showed that using alpha-divergence in calibration provides better calibrated uncertainty estimates for specific alpha choices and is more efficient, especially for complex architectures, as empirically demonstrated in regression problems.
Bayesian Neural Networks (BNNs) often result uncalibrated after training, usually tending towards overconfidence. Devising effective calibration methods with low impact in terms of computational complexity is thus of central interest. In this paper we present calibration methods for BNNs based on the alpha divergences from Information Geometry. We compare the use of alpha divergence in training and in calibration, and we show how the use in calibration provides better calibrated uncertainty estimates for specific choices of alpha and is more efficient especially for complex network architectures. We empirically demonstrate the advantages of alpha calibration in regression problems involving parameter estimation and inferred correlations between output uncertainties.