On Representations of Mean-Field Variational Inference
This provides theoretical tools to analyze and ensure convergence of MFVI algorithms, which is incremental but important for improving reliability in variational inference methods used in machine learning.
The paper tackles the analysis of mean-field variational inference (MFVI) algorithms by developing a framework that represents MFVI as a gradient flow on Wasserstein space, a system of Fokker-Planck-like equations, and a diffusion process, establishing rigorous convergence guarantees for time-discretized implementations.
The mean field variational inference (MFVI) formulation restricts the general Bayesian inference problem to the subspace of product measures. We present a framework to analyze MFVI algorithms, which is inspired by a similar development for general variational Bayesian formulations. Our approach enables the MFVI problem to be represented in three different manners: a gradient flow on Wasserstein space, a system of Fokker-Planck-like equations and a diffusion process. Rigorous guarantees are established to show that a time-discretized implementation of the coordinate ascent variational inference algorithm in the product Wasserstein space of measures yields a gradient flow in the limit. A similar result is obtained for their associated densities, with the limit being given by a quasi-linear partial differential equation. A popular class of practical algorithms falls in this framework, which provides tools to establish convergence. We hope this framework could be used to guarantee convergence of algorithms in a variety of approaches, old and new, to solve variational inference problems.