Dimension-Free Multimodal Sampling via Preconditioned Annealed Langevin Dynamics
This work bridges the gap between empirical success and theoretical understanding of annealed Langevin dynamics for multimodal sampling, offering guarantees that hold uniformly across dimensions, which is crucial for high-dimensional applications.
The paper provides a uniform-in-dimension analysis of annealed Langevin dynamics for Gaussian-mixture targets, proving that under spectral conditions on the smoothing covariance and with appropriate preconditioning, the algorithm achieves a prescribed accuracy in KL divergence within a dimension-uniform time horizon, even with imperfect initialization and approximate scores.
Designing sampling algorithms for multimodal targets that remain stable under refinement of the finite-dimensional approximation of an underlying function-space problem is a central challenge. Annealed Langevin dynamics (ALD) is a natural alternative to classical Langevin in this context, since it is often observed to improve exploration across modes. Yet a gap remains between its empirical success and existing theory: under which conditions can ALD be guaranteed to remain stable across dimensions? In this paper, we bridge this gap by providing a uniform-in-dimension analysis of continuous-time ALD for Gaussian-mixture targets. Along an explicit annealing path obtained by gradually removing Gaussian smoothing from the target, we identify spectral conditions linking the smoothing covariance to the component covariances under which ALD achieves a prescribed accuracy in Kullback-Leibler divergence within a dimension-uniform time horizon. We then establish stability in a perturbative regime with imperfect initialization and approximate scores. Under a misspecified-mixture score model, we show that preconditioning ALD with an operator whose spectrum decays sufficiently fast prevents error terms from accumulating across coordinates and thereby preserves dimension-uniform control.