Stochastic Gradient Langevin Dynamics with Variance Reduction
This work addresses optimization challenges in nonconvex settings for machine learning researchers, presenting incremental improvements to SGLD.
The paper tackles the problem of improving convergence to local minimizers of nonconvex objective functions using Stochastic Gradient Langevin Dynamics (SGLD) accelerated by variance reduction, proving improved convergence properties and ergodicity for potential global optimization.
Stochastic gradient Langevin dynamics (SGLD) has gained the attention of optimization researchers due to its global optimization properties. This paper proves an improved convergence property to local minimizers of nonconvex objective functions using SGLD accelerated by variance reductions. Moreover, we prove an ergodicity property of the SGLD scheme, which gives insights on its potential to find global minimizers of nonconvex objectives.