Projection-free nonconvex stochastic optimization on Riemannian manifolds
This work addresses optimization problems with manifold constraints for researchers in optimization and machine learning, representing an incremental extension of existing methods to Riemannian settings.
The authors tackled constrained optimization of smooth functions on Riemannian manifolds by introducing stochastic projection-free Frank-Wolfe methods for nonconvex and geodesically convex problems, achieving convergence rates comparable to Euclidean counterparts and state-of-the-art empirical performance on tasks like computing Karcher means and Wasserstein barycenters.
We study stochastic projection-free methods for constrained optimization of smooth functions on Riemannian manifolds, i.e., with additional constraints beyond the parameter domain being a manifold. Specifically, we introduce stochastic Riemannian Frank-Wolfe methods for nonconvex and geodesically convex problems. We present algorithms for both purely stochastic optimization and finite-sum problems. For the latter, we develop variance-reduced methods, including a Riemannian adaptation of the recently proposed Spider technique. For all settings, we recover convergence rates that are comparable to the best-known rates for their Euclidean counterparts. Finally, we discuss applications to two classic tasks: The computation of the Karcher mean of positive definite matrices and Wasserstein barycenters for multivariate normal distributions. For both tasks, stochastic Fw methods yield state-of-the-art empirical performance.