R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate
This work provides a faster optimization algorithm for problems in machine learning and related fields that involve Riemannian manifolds, representing an incremental improvement over prior methods.
The authors tackled stochastic optimization on Riemannian manifolds by adapting the SPIDER algorithm, achieving faster convergence rates than existing methods in both finite sum and stochastic settings, with curvature-independent rates for nonconvex and strongly convex cases.
We study smooth stochastic optimization problems on Riemannian manifolds. Via adapting the recently proposed SPIDER algorithm \citep{fang2018spider} (a variance reduced stochastic method) to Riemannian manifold, we can achieve faster rate than known algorithms in both the finite sum and stochastic settings. Unlike previous works, by \emph{not} resorting to bounding iterate distances, our analysis yields curvature independent convergence rates for both the nonconvex and strongly convex cases.