Implicit Riemannian Optimism with Applications to Min-Max Problems
This work addresses optimization challenges in non-Euclidean spaces for researchers in machine learning and optimization, offering incremental improvements with specific theoretical guarantees.
The authors tackled the problem of online learning and min-max optimization on Hadamard manifolds by introducing a Riemannian optimistic algorithm that handles in-manifold constraints and matches Euclidean regret bounds without geometric dependencies, and developed methods for min-max problems that nearly achieve the lower-bound gradient oracle complexity for the first time.
We introduce a Riemannian optimistic online learning algorithm for Hadamard manifolds based on inexact implicit updates. Unlike prior work, our method can handle in-manifold constraints, and matches the best known regret bounds in the Euclidean setting with no dependence on geometric constants, like the minimum curvature. Building on this, we develop algorithms for g-convex, g-concave smooth min-max problems on Hadamard manifolds. Notably, one method nearly matches the gradient oracle complexity of the lower bound for Euclidean problems, for the first time.