Stochastic Gradient Descent in Continuous Time: A Central Limit Theorem
This provides theoretical guarantees for efficient statistical learning in continuous-time models used in science, engineering, and finance, but it is incremental as it extends existing analysis to new conditions.
The paper tackled the asymptotic convergence rate of Stochastic Gradient Descent in Continuous Time (SGDCT) by proving a central limit theorem for strongly convex and non-convex objective functions, with an $L^{p}$ convergence rate also established for the strongly convex case.
Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter updates occur in continuous time and satisfy a stochastic differential equation. This paper analyzes the asymptotic convergence rate of the SGDCT algorithm by proving a central limit theorem (CLT) for strongly convex objective functions and, under slightly stronger conditions, for non-convex objective functions as well. An $L^{p}$ convergence rate is also proven for the algorithm in the strongly convex case. The mathematical analysis lies at the intersection of stochastic analysis and statistical learning.