Convergence rates for the stochastic gradient descent method for non-convex objective functions
This work addresses the theoretical understanding of optimization algorithms for non-convex problems, which is incremental as it extends existing convergence analyses to more general settings.
The paper tackles the problem of analyzing convergence rates for stochastic gradient descent (SGD) applied to non-convex objective functions, proving local convergence to minima and providing rate estimates, with applicability to machine learning contexts.
We prove the local convergence to minima and estimates on the rate of convergence for the stochastic gradient descent method in the case of not necessarily globally convex nor contracting objective functions. In particular, the results are applicable to simple objective functions arising in machine learning.