Variational Inference with Locally Enhanced Bounds for Hierarchical Models
This addresses scalability and accuracy issues in hierarchical model inference for practitioners in fields like Bayesian statistics and machine learning, though it appears incremental as an enhancement to existing variational methods.
The paper tackles the challenge of accurate variational inference in hierarchical models by proposing a new family of variational bounds that apply tightening methods separately for each group of local variables, resulting in better posterior approximations than baselines.
Hierarchical models represent a challenging setting for inference algorithms. MCMC methods struggle to scale to large models with many local variables and observations, and variational inference (VI) may fail to provide accurate approximations due to the use of simple variational families. Some variational methods (e.g. importance weighted VI) integrate Monte Carlo methods to give better accuracy, but these tend to be unsuitable for hierarchical models, as they do not allow for subsampling and their performance tends to degrade for high dimensional models. We propose a new family of variational bounds for hierarchical models, based on the application of tightening methods (e.g. importance weighting) separately for each group of local random variables. We show that our approach naturally allows the use of subsampling to get unbiased gradients, and that it fully leverages the power of methods that build tighter lower bounds by applying them independently in lower dimensional spaces, leading to better results and more accurate posterior approximations than relevant baselines.