Gaussian variational approximation with sparse precision matrices
This work addresses variational inference for high-dimensional Bayesian models, offering a more efficient and stable method, though it appears incremental as it builds on existing sparse precision and gradient optimization techniques.
The authors tackled the problem of learning Gaussian variational approximations for high-dimensional posteriors by imposing sparsity in precision matrices to reflect conditional independence, achieving flexibility and parsimony through Cholesky factor parameterization. They developed efficient stochastic gradient methods with lower-variance estimators, illustrating the approach on generalized linear mixed models and state space models.
We consider the problem of learning a Gaussian variational approximation to the posterior distribution for a high-dimensional parameter, where we impose sparsity in the precision matrix to reflect appropriate conditional independence structure in the model. Incorporating sparsity in the precision matrix allows the Gaussian variational distribution to be both flexible and parsimonious, and the sparsity is achieved through parameterization in terms of the Cholesky factor. Efficient stochastic gradient methods which make appropriate use of gradient information for the target distribution are developed for the optimization. We consider alternative estimators of the stochastic gradients which have lower variation and are more stable. Our approach is illustrated using generalized linear mixed models and state space models for time series.