Mirror Descent on Riemannian Manifolds
This work addresses scalable optimization on manifolds for applications like image processing and neural networks, representing an incremental extension of existing methods.
The paper generalizes Mirror Descent to Riemannian manifolds, developing a Riemannian Mirror Descent framework with stochastic variants and establishing non-asymptotic convergence guarantees.
Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD. As an application to the Stiefel manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, we obtain a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.