Wasserstein Contraction of Coordinate Ascent Variational Inference
Provides theoretical convergence guarantees for a widely used variational inference algorithm, benefiting practitioners in Bayesian statistics and machine learning.
The paper proves that coordinate ascent variational inference contracts in Wasserstein distance under certain conditions, providing local convergence guarantees for general smooth manifolds and some non-smooth spaces, with applications to Bayesian Gaussian mixture models and high-dimensional probit/logistic regression.
We study the contraction in Wasserstein distance of the coordinate ascent variational inference algorithm. This is shown to hold under a transport-information inequality at the fixed points and a functional smoothness condition. The results are general and sharp, allow for local convergence guarantees, hold for general smooth manifolds, and also in some non-smooth spaces. We consider applications to Bayesian Gaussian Mixture Models, and high-dimensional Bayesian Probit Regression, and Logistic Regression with Pólya-Gamma random variables (i.e. Jaakkola-Jordan's algorithm).