A Riemannian ADMM
This addresses a gap in optimization methods for machine learning tasks like sparse PCA, but it is incremental as it extends existing ADMM frameworks to a specific nonconvex setting.
The paper tackles optimization problems on Riemannian manifolds with nonsmooth objectives, proposing a Riemannian ADMM algorithm that is the first to handle nonsmooth functions over such nonconvex sets, with demonstrated advantages in numerical experiments.
We consider a class of Riemannian optimization problems where the objective is the sum of a smooth function and a nonsmooth function, considered in the ambient space. This class of problems finds important applications in machine learning and statistics such as the sparse principal component analysis, sparse spectral clustering, and orthogonal dictionary learning. We propose a Riemannian alternating direction method of multipliers (ADMM) to solve this class of problems. Our algorithm adopts easily computable steps in each iteration. The iteration complexity of the proposed algorithm for obtaining an $ε$-stationary point is analyzed under mild assumptions. Existing ADMM for solving nonconvex problems either does not allow nonconvex constraint set, or does not allow nonsmooth objective function. Our algorithm is the first ADMM type algorithm that minimizes a nonsmooth objective over manifold -- a particular nonconvex set. Numerical experiments are conducted to demonstrate the advantage of the proposed method.