Uncertainty-Aware (UNA) Bases for Deep Bayesian Regression Using Multi-Headed Auxiliary Networks
This work addresses uncertainty calibration for deep Bayesian models, which is crucial for safe deployment in risk-sensitive domains, though it appears incremental as it builds on existing NLMs.
The paper tackled the problem of Neural Linear Models (NLMs) underestimating uncertainty on out-of-distribution inputs, which limits their use in risk-sensitive applications, and proposed a novel training framework to capture useful predictive uncertainties.
Neural Linear Models (NLM) are deep Bayesian models that produce predictive uncertainties by learning features from the data and then performing Bayesian linear regression over these features. Despite their popularity, few works have focused on methodically evaluating the predictive uncertainties of these models. In this work, we demonstrate that traditional training procedures for NLMs drastically underestimate uncertainty on out-of-distribution inputs, and that they therefore cannot be naively deployed in risk-sensitive applications. We identify the underlying reasons for this behavior and propose a novel training framework that captures useful predictive uncertainties for downstream tasks.