Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles
This addresses the need for reliable uncertainty estimation in deep learning for tasks such as confidence assessment or abstention decisions, representing an incremental improvement over existing methods.
The paper tackles the problem of efficient uncertainty estimation in Bayesian deep learning by introducing scalable Bayesian Monte Carlo (SBMC), which achieves comparable or better accuracy and substantially improved uncertainty quantification, especially for epistemic uncertainty, on datasets like MNIST, CIFAR, and IMDb within the same wall-clock time as state-of-the-art methods like deep ensembles.
This work introduces a new method designed for Bayesian deep learning called scalable Bayesian Monte Carlo (SBMC). The method is comprised of a model and an algorithm. The model interpolates between a point estimator and the posterior. The algorithm is a parallel implementation of sequential Monte Carlo sampler (SMC$_\parallel$) or Markov chain Monte Carlo (MCMC$_\parallel$). We collectively refer to these consistent (asymptotically unbiased) algorithms as Bayesian Monte Carlo (BMC), and any such algorithm can be used in our SBMC method. The utility of the method is demonstrated on practical examples: MNIST, CIFAR, IMDb. A systematic numerical study reveals that for the same wall-clock time as state-of-the-art (SOTA) methods like deep ensembles (DE), SBMC achieves comparable or better accuracy and substantially improved uncertainty quantification (UQ)--in particular, epistemic UQ. This is demonstrated on the downstream task of estimating the confidence in predictions, which can be used for reliability assessment or abstention decisions.