Bayesian Computation in Deep Learning
It provides an introductory overview for researchers and practitioners in deep learning, but is incremental as it synthesizes existing methods without new contributions.
The paper reviews approximate Bayesian inference techniques, specifically stochastic gradient Markov chain Monte Carlo and variational inference, for improving uncertainty estimation in Bayesian neural networks and latent variable inference in deep generative models.
Bayesian methods have shown success in deep learning applications. For example, in predictive tasks, Bayesian neural networks leverage Bayesian reasoning of model uncertainty to improve the reliability and uncertainty awareness of deep neural networks. In generative modeling domain, many widely used deep generative models, such as deep latent variable models, require approximate Bayesian inference to infer their latent variables for the training. In this chapter, we provide an introduction to approximate inference techniques as Bayesian computation methods applied to deep learning models, with a focus on Bayesian neural networks and deep generative models. We review two arguably most popular approximate Bayesian computational methods, stochastic gradient Markov chain Monte Carlo (SG-MCMC) and variational inference (VI), and explain their unique challenges in posterior inference as well as the solutions when applied to deep learning models.