On the Curved Geometry of Accelerated Optimization
This work provides a novel geometric framework for understanding accelerated optimization, which is incremental but offers insights for researchers in optimization theory.
The authors tackled the problem of understanding Nesterov's accelerated gradient method by proposing a differential geometric interpretation, showing it as a proximal point method on a Riemannian manifold, and analyzed its convergence rate for quadratic objectives, achieving results consistent with known rates.
In this work we propose a differential geometric motivation for Nesterov's accelerated gradient method (AGM) for strongly-convex problems. By considering the optimization procedure as occurring on a Riemannian manifold with a natural structure, The AGM method can be seen as the proximal point method applied in this curved space. This viewpoint can also be extended to the continuous time case, where the accelerated gradient method arises from the natural block-implicit Euler discretization of an ODE on the manifold. We provide an analysis of the convergence rate of this ODE for quadratic objectives.