Subspace Langevin Monte Carlo
This addresses computational bottlenecks in data science and machine learning for sampling tasks, though it appears incremental as it generalizes existing Langevin Monte Carlo variants.
The paper tackles the problem of sampling from high-dimensional distributions, which is computationally challenging, by introducing Subspace Langevin Monte Carlo (SLMC), a method that projects updates onto subsampled eigenblocks, resulting in superior adaptability and efficiency compared to traditional methods, with error guarantees and experimental validation on ill-conditioned distributions.
Sampling from high-dimensional distributions has wide applications in data science and machine learning but poses significant computational challenges. We introduce Subspace Langevin Monte Carlo (SLMC), a novel and efficient sampling method that generalizes random-coordinate Langevin Monte Carlo and preconditioned Langevin Monte Carlo by projecting the Langevin update onto subsampled eigenblocks of a time-varying preconditioner at each iteration. The advantage of SLMC is its superior adaptability and computational efficiency compared to traditional Langevin Monte Carlo and preconditioned Langevin Monte Carlo. Using coupling arguments, we establish error guarantees for SLMC and demonstrate its practical effectiveness through a few experiments on sampling from ill-conditioned distributions.