A Particle Algorithm for Mean-Field Variational Inference
This provides a more flexible and theoretically grounded method for posterior inference in statistics and machine learning, though it is incremental as it builds on existing variational inference frameworks.
The paper tackles the challenge of mean-field variational inference (MFVI) by introducing a particle-based algorithm called PArticle VI (PAVI), which eliminates the need for parametric assumptions on complete conditionals and applies to nonparametric settings, with results including a non-asymptotic finite-particle convergence guarantee.
Variational inference is a fast and scalable alternative to Markov chain Monte Carlo and has been widely applied to posterior inference tasks in statistics and machine learning. A traditional approach for implementing mean-field variational inference (MFVI) is coordinate ascent variational inference (CAVI), which relies crucially on parametric assumptions on complete conditionals. In this paper, we introduce a novel particle-based algorithm for mean-field variational inference, which we term PArticle VI (PAVI). Notably, our algorithm does not rely on parametric assumptions on complete conditionals, and it applies to the nonparametric setting. We provide non-asymptotic finite-particle convergence guarantee for our algorithm. To our knowledge, this is the first end-to-end guarantee for particle-based MFVI.