Approximation Properties of Variational Bayes for Vector Autoregressions
This provides theoretical insights into VB's performance for statisticians and practitioners using vector autoregressions, but it is incremental as it focuses on a specific model.
The paper tackles the problem of unknown approximation error in Variational Bayes (VB) for Bayesian inference by deriving the error for linear Gaussian multi-equation regression, showing that VB approximates the posterior mean perfectly and accurately estimates predictive densities.
Variational Bayes (VB) is a recent approximate method for Bayesian inference. It has the merit of being a fast and scalable alternative to Markov Chain Monte Carlo (MCMC) but its approximation error is often unknown. In this paper, we derive the approximation error of VB in terms of mean, mode, variance, predictive density and KL divergence for the linear Gaussian multi-equation regression. Our results indicate that VB approximates the posterior mean perfectly. Factors affecting the magnitude of underestimation in posterior variance and mode are revealed. Importantly, We demonstrate that VB estimates predictive densities accurately.