The variational Laplace approach to approximate Bayesian inference
This is an incremental contribution that refines existing methods for researchers in Bayesian statistics and machine learning.
The paper reviews variational-Laplace (VL) schemes for approximate Bayesian inference, deriving new theoretical results on asymptotic convergence and proposing extensions to hierarchical generative models with hyperparameters.
Variational approaches to approximate Bayesian inference provide very efficient means of performing parameter estimation and model selection. Among these, so-called variational-Laplace or VL schemes rely on Gaussian approximations to posterior densities on model parameters. In this note, we review the main variants of VL approaches, that follow from considering nonlinear models of continuous and/or categorical data. En passant, we also derive a few novel theoretical results that complete the portfolio of existing analyses of variational Bayesian approaches, including investigations of their asymptotic convergence. We also suggest practical ways of extending existing VL approaches to hierarchical generative models that include (e.g., precision) hyperparameters.