Variational Bayes: A report on approaches and applications
This is an incremental review paper summarizing existing methods for uncertainty estimation in deep learning, with no new results presented.
The report reviews variational inference concepts for Bayesian neural networks to address uncertainty quantification, comparing approximation methods and discussing applications in reinforcement and continual learning.
Deep neural networks have achieved impressive results on a wide variety of tasks. However, quantifying uncertainty in the network's output is a challenging task. Bayesian models offer a mathematical framework to reason about model uncertainty. Variational methods have been used for approximating intractable integrals that arise in Bayesian inference for neural networks. In this report, we review the major variational inference concepts pertinent to Bayesian neural networks and compare various approximation methods used in literature. We also talk about the applications of variational bayes in Reinforcement learning and continual learning.