Forward $χ^2$ Divergence Based Variational Importance Sampling
This addresses a key limitation in variational inference for machine learning practitioners working with latent variable models, offering improved performance, though it appears incremental as an enhancement to existing methods.
The paper tackles the challenge of achieving high log-likelihood in latent variable models with complex posteriors by introducing a variational importance sampling method that directly maximizes log-likelihood using forward χ² divergence. Results show it consistently outperforms state-of-the-art baselines in log-likelihood and parameter estimation.
Maximizing the log-likelihood is a crucial aspect of learning latent variable models, and variational inference (VI) stands as the commonly adopted method. However, VI can encounter challenges in achieving a high log-likelihood when dealing with complicated posterior distributions. In response to this limitation, we introduce a novel variational importance sampling (VIS) approach that directly estimates and maximizes the log-likelihood. VIS leverages the optimal proposal distribution, achieved by minimizing the forward $χ^2$ divergence, to enhance log-likelihood estimation. We apply VIS to various popular latent variable models, including mixture models, variational auto-encoders, and partially observable generalized linear models. Results demonstrate that our approach consistently outperforms state-of-the-art baselines, both in terms of log-likelihood and model parameter estimation.