Constrained Langevin Algorithms with L-mixing External Random Variables
This provides a theoretical improvement for constrained optimization and sampling in machine learning, though it is incremental as it builds on existing Langevin algorithm analysis.
The paper tackles the problem of analyzing constrained Langevin algorithms for non-convex learning with L-mixing data variables, achieving a deviation of O(T^{-1/2} log T) in 1-Wasserstein distance, which improves on previous bounds for constrained problems and matches the best-known bound for unconstrained ones.
Langevin algorithms are gradient descent methods augmented with additive noise, and are widely used in Markov Chain Monte Carlo (MCMC) sampling, optimization, and machine learning. In recent years, the non-asymptotic analysis of Langevin algorithms for non-convex learning has been extensively explored. For constrained problems with non-convex losses over a compact convex domain with IID data variables, the projected Langevin algorithm achieves a deviation of $O(T^{-1/4} (\log T)^{1/2})$ from its target distribution [27] in $1$-Wasserstein distance. In this paper, we obtain a deviation of $O(T^{-1/2} \log T)$ in $1$-Wasserstein distance for non-convex losses with $L$-mixing data variables and polyhedral constraints (which are not necessarily bounded). This improves on the previous bound for constrained problems and matches the best-known bound for unconstrained problems.