Riemannian accelerated gradient methods via extrapolation
This work addresses optimization on manifolds for researchers in machine learning and optimization, offering an incremental improvement with practical benefits.
The authors tackled the problem of accelerating Riemannian gradient methods on manifolds by proposing an extrapolation-based scheme, achieving asymptotically optimal convergence rates and demonstrating computational advantages over existing Riemannian Nesterov methods in experiments.
In this paper, we propose a simple acceleration scheme for Riemannian gradient methods by extrapolating iterates on manifolds. We show when the iterates are generated from Riemannian gradient descent method, the accelerated scheme achieves the optimal convergence rate asymptotically and is computationally more favorable than the recently proposed Riemannian Nesterov accelerated gradient methods. Our experiments verify the practical benefit of the novel acceleration strategy.