Curvature-Dependant Global Convergence Rates for Optimization on Manifolds of Bounded Geometry
This work addresses convergence guarantees for optimization on manifolds, which is incremental as it refines existing bounds and applies to specific domains like matrix groups and Grassmannians.
The paper tackles the problem of optimizing weakly convex functions on manifolds with bounded geometry by providing curvature-dependent convergence rates for Riemannian gradient descent and the dynamic trivialization algorithm, achieving tighter bounds on the Hessian norm of the Riemannian exponential and computing explicit rates for manifolds like the special orthogonal group and real Grassmannian.
We give curvature-dependant convergence rates for the optimization of weakly convex functions defined on a manifold of 1-bounded geometry via Riemannian gradient descent and via the dynamic trivialization algorithm. In order to do this, we give a tighter bound on the norm of the Hessian of the Riemannian exponential than the previously known. We compute these bounds explicitly for some manifolds commonly used in the optimization literature such as the special orthogonal group and the real Grassmannian. Along the way, we present self-contained proofs of fully general bounds on the norm of the differential of the exponential map and certain cosine inequalities on manifolds, which are commonly used in optimization on manifolds.