Local convexity of the TAP free energy and AMP convergence for Z2-synchronization
This work provides a rigorous foundation for variational inference in high-dimensional Bayesian models, addressing convergence issues for researchers in statistics and machine learning, though it is incremental as it builds on existing TAP and AMP frameworks.
The paper tackles the problem of mean-field variational Bayesian inference for Z2-synchronization by proving that the TAP free energy has a unique local minimizer near the Bayes posterior mean for any signal strength above the weak-recovery threshold, with strong local convexity ensuring linear convergence of a natural-gradient algorithm from a local initialization. It also shows that Approximate Message Passing (AMP) is asymptotically stable and linearly convergent to this minimizer for sufficiently large signal strengths, providing stronger guarantees than previous state evolution analyses.
We study mean-field variational Bayesian inference using the TAP approach, for Z2-synchronization as a prototypical example of a high-dimensional Bayesian model. We show that for any signal strength $λ> 1$ (the weak-recovery threshold), there exists a unique local minimizer of the TAP free energy functional near the mean of the Bayes posterior law. Furthermore, the TAP free energy in a local neighborhood of this minimizer is strongly convex. Consequently, a natural-gradient/mirror-descent algorithm achieves linear convergence to this minimizer from a local initialization, which may be obtained by a constant number of iterates of Approximate Message Passing (AMP). This provides a rigorous foundation for variational inference in high dimensions via minimization of the TAP free energy. We also analyze the finite-sample convergence of AMP, showing that AMP is asymptotically stable at the TAP minimizer for any $λ> 1$, and is linearly convergent to this minimizer from a spectral initialization for sufficiently large $λ$. Such a guarantee is stronger than results obtainable by state evolution analyses, which only describe a fixed number of AMP iterations in the infinite-sample limit. Our proofs combine the Kac-Rice formula and Sudakov-Fernique Gaussian comparison inequality to analyze the complexity of critical points that satisfy strong convexity and stability conditions within their local neighborhoods.