When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint
This addresses a fundamental gap in MCMC sampling theory by enabling dimension-free convergence for specific convex problems, which is incremental as it applies to particular classes rather than all cases.
The paper tackles the long-standing open problem of whether Langevin algorithms can achieve dimension-independent convergence rates, providing an affirmative answer for large classes of Lipschitz or smooth convex problems with normal priors by viewing Langevin algorithm as composite optimization and developing a new analysis technique.
There has been a surge of works bridging MCMC sampling and optimization, with a specific focus on translating non-asymptotic convergence guarantees for optimization problems into the analysis of Langevin algorithms in MCMC sampling. A conspicuous distinction between the convergence analysis of Langevin sampling and that of optimization is that all known convergence rates for Langevin algorithms depend on the dimensionality of the problem, whereas the convergence rates for optimization are dimension-free for convex problems. Whether a dimension independent convergence rate can be achieved by Langevin algorithm is thus a long-standing open problem. This paper provides an affirmative answer to this problem for large classes of either Lipschitz or smooth convex problems with normal priors. By viewing Langevin algorithm as composite optimization, we develop a new analysis technique that leads to dimension independent convergence rates for such problems.