Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization
This provides theoretical guarantees for scalable Bayesian inference and nonconvex optimization, addressing a bottleneck in machine learning, but it is incremental as it builds on prior SGHMC analysis.
The paper tackles the problem of analyzing the convergence of Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) for nonconvex optimization and sampling, proving that the Wasserstein-2 distance to the target measure is uniformly controlled by step-size, enabling high-precision results and best-known convergence rates to global minima.
We provide a nonasymptotic analysis of the convergence of the stochastic gradient Hamiltonian Monte Carlo (SGHMC) to a target measure in Wasserstein-2 distance without assuming log-concavity. Our analysis quantifies key theoretical properties of the SGHMC as a sampler under local conditions which significantly improves the findings of previous results. In particular, we prove that the Wasserstein-2 distance between the target and the law of the SGHMC is uniformly controlled by the step-size of the algorithm, therefore demonstrate that the SGHMC can provide high-precision results uniformly in the number of iterations. The analysis also allows us to obtain nonasymptotic bounds for nonconvex optimization problems under local conditions and implies that the SGHMC, when viewed as a nonconvex optimizer, converges to a global minimum with the best known rates. We apply our results to obtain nonasymptotic bounds for scalable Bayesian inference and nonasymptotic generalization bounds.