Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization
This work provides theoretical guarantees for SGLD in nonconvex settings, which is important for practitioners in machine learning and optimization, though it is incremental as it relaxes existing assumptions.
The paper tackled the problem of non-asymptotic analysis for Stochastic Gradient Langevin Dynamics (SGLD) in nonconvex optimization, obtaining convergence estimates in Wasserstein distances and an error bound for expected excess risk under relaxed local conditions, with examples in variational inference and index tracking optimization.
In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the aforementioned Wasserstein-2 convergence result can be applied to establish a non-asymptotic error bound for the expected excess risk. Crucially, these results are obtained under a local Lipschitz condition and a local dissipativity condition where we remove the uniform dependence in the data stream. We illustrate the importance of this relaxation by presenting examples from variational inference and from index tracking optimization.