Variational Boosting: Iteratively Refining Posterior Approximations
This work addresses the challenge of improving posterior approximations in statistical models for practitioners, though it appears incremental as it builds on existing variational inference methods.
The authors tackled the problem of approximating intractable distributions in variational inference by proposing variational boosting, a method that iteratively refines posterior approximations to improve accuracy with increased computation, and showed it outperforms existing algorithms in accuracy and efficiency on synthetic and real models.
We propose a black-box variational inference method to approximate intractable distributions with an increasingly rich approximating class. Our method, termed variational boosting, iteratively refines an existing variational approximation by solving a sequence of optimization problems, allowing the practitioner to trade computation time for accuracy. We show how to expand the variational approximating class by incorporating additional covariance structure and by introducing new components to form a mixture. We apply variational boosting to synthetic and real statistical models, and show that resulting posterior inferences compare favorably to existing posterior approximation algorithms in both accuracy and efficiency.