Accelerated Methods for Riemannian Min-Max Optimization Ensuring Bounded Geometric Penalties
This work addresses optimization challenges in machine learning and related fields by providing more efficient algorithms for Riemannian min-max problems, though it is incremental as it builds on and improves existing methods.
The paper tackles the problem of min-max optimization on Riemannian manifolds by designing accelerated methods that ensure bounded geometric penalties, achieving global linear convergence and reducing geometric constants without requiring iterates to stay in a pre-specified compact set.
In this work, we study optimization problems of the form $\min_x \max_y f(x, y)$, where $f(x, y)$ is defined on a product Riemannian manifold $\mathcal{M} \times \mathcal{N}$ and is $μ_x$-strongly geodesically convex (g-convex) in $x$ and $μ_y$-strongly g-concave in $y$, for $μ_x, μ_y \geq 0$. We design accelerated methods when $f$ is $(L_x, L_y, L_{xy})$-smooth and $\mathcal{M}$, $\mathcal{N}$ are Hadamard. To that aim we introduce new g-convex optimization results, of independent interest: we show global linear convergence for metric-projected Riemannian gradient descent and improve existing accelerated methods by reducing geometric constants. Additionally, we complete the analysis of two previous works applying to the Riemannian min-max case by removing an assumption about iterates staying in a pre-specified compact set.