Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold
It addresses decentralized optimization for non-convex problems on manifolds, which is incremental as it extends conjugate gradient methods to distributed scenarios.
The paper tackles the problem of distributed optimization on the Stiefel manifold by proposing a decentralized Riemannian conjugate gradient method (DRCGD) that avoids expensive geometric operations, achieving global convergence as the first such algorithm in this setting.
The conjugate gradient method is a crucial first-order optimization method that generally converges faster than the steepest descent method, and its computational cost is much lower than that of second-order methods. However, while various types of conjugate gradient methods have been studied in Euclidean spaces and on Riemannian manifolds, there is little study for those in distributed scenarios. This paper proposes a decentralized Riemannian conjugate gradient descent (DRCGD) method that aims at minimizing a global function over the Stiefel manifold. The optimization problem is distributed among a network of agents, where each agent is associated with a local function, and the communication between agents occurs over an undirected connected graph. Since the Stiefel manifold is a non-convex set, a global function is represented as a finite sum of possibly non-convex (but smooth) local functions. The proposed method is free from expensive Riemannian geometric operations such as retractions, exponential maps, and vector transports, thereby reducing the computational complexity required by each agent. To the best of our knowledge, DRCGD is the first decentralized Riemannian conjugate gradient algorithm to achieve global convergence over the Stiefel manifold.