Shifted Composition IV: Toward Ballistic Acceleration for Log-Concave Sampling
This work addresses a major open problem in computational statistics and machine learning by enabling faster sampling algorithms for log-concave distributions, which is incremental but crucial for applications like Bayesian inference.
The paper tackles the challenge of achieving accelerated convergence for log-concave sampling, analogous to convex optimization, by developing a new coupling-based error analysis framework for Underdamped Langevin Dynamics (ULD). It establishes the first ballistic acceleration result with sublinear dependence on condition number and provides a dimension-dependent iteration complexity of $d^{1/3}$ for sampling to constant total variation error.
Acceleration is a celebrated cornerstone of convex optimization, enabling gradient-based algorithms to converge sublinearly in the condition number. A major open question is whether an analogous acceleration phenomenon is possible for log-concave sampling. Underdamped Langevin dynamics (ULD) has long been conjectured to be the natural candidate for acceleration, but a central challenge is that its degeneracy necessitates the development of new analysis approaches, e.g., the theory of hypocoercivity. Although recent breakthroughs established ballistic acceleration for the (continuous-time) ULD diffusion via space-time Poincare inequalities, (discrete-time) algorithmic results remain entirely open: the discretization error of existing analysis techniques dominates any continuous-time acceleration. In this paper, we give a new coupling-based local error framework for analyzing ULD and its numerical discretizations in KL divergence. This extends the framework in Shifted Composition III from uniformly elliptic diffusions to degenerate diffusions, and shares its virtues: the framework is user-friendly, applies to sophisticated discretization schemes, and does not require contractivity. Applying this framework to the randomized midpoint discretization of ULD establishes the first ballistic acceleration result for log-concave sampling (i.e., sublinear dependence on the condition number). Along the way, we also obtain the first $d^{1/3}$ iteration complexity guarantee for sampling to constant total variation error in dimension $d$.