Global Riemannian Acceleration in Hyperbolic and Spherical Spaces
This work provides foundational optimization methods for machine learning models operating in non-Euclidean geometric spaces, which are increasingly relevant for tasks like embedding and representation learning.
This paper introduces the first globally accelerated first-order optimization methods for L-smooth and geodesically convex or strongly geodesically convex functions on hyperbolic and spherical spaces. These methods achieve Euclidean-equivalent acceleration rates, up to logarithmic factors and an additional factor dependent on initial distance and curvature.
We further research on the accelerated optimization phenomenon on Riemannian manifolds by introducing accelerated global first-order methods for the optimization of $L$-smooth and geodesically convex (g-convex) or $μ$-strongly g-convex functions defined on the hyperbolic space or a subset of the sphere. For a manifold other than the Euclidean space, these are the first methods to \emph{globally} achieve the same rates as accelerated gradient descent in the Euclidean space with respect to $L$ and $ε$ (and $μ$ if it applies), up to log factors. Due to the geometric deformations, our rates have an extra factor, depending on the initial distance $R$ to a minimizer and the curvature $K$, with respect to Euclidean accelerated algorithms As a proxy for our solution, we solve a constrained non-convex Euclidean problem, under a condition between convexity and \emph{quasar-convexity}, of independent interest. Additionally, for any Riemannian manifold of bounded sectional curvature, we provide reductions from optimization methods for smooth and g-convex functions to methods for smooth and strongly g-convex functions and vice versa. We also reduce global optimization to optimization over bounded balls where the effect of the curvature is reduced.