Riemannian Federated Learning via Averaging Gradient Streams
This work addresses federated learning challenges in non-Euclidean spaces, which is incremental as it extends existing FL methods to the Riemannian setting.
The paper tackles federated learning on Riemannian manifolds by proposing RFedAGS, an algorithm that handles partial participation and data heterogeneity, achieving global convergence with sublinear rates under decaying step sizes and convergence to a neighborhood of stationary points under fixed step sizes, as validated by experiments on synthetic and real-world data.
Federated learning (FL) as a distributed learning paradigm has a significant advantage in addressing large-scale machine learning tasks. In the Euclidean setting, FL algorithms have been extensively studied with both theoretical and empirical success. However, there exist few works that investigate federated learning algorithms in the Riemannian setting. In particular, critical challenges such as partial participation and data heterogeneity among agents are not explored in the Riemannian federated setting. This paper presents and analyzes a Riemannian FL algorithm, called RFedAGS, based on a new efficient server aggregation -- averaging gradient streams, which can simultaneously handle partial participation and data heterogeneity. We theoretically show that the proposed RFedAGS has global convergence and sublinear convergence rate under decaying step sizes cases; and converges sublinearly/linearly to a neighborhood of a stationary point/solution under fixed step sizes cases. These analyses are based on a vital and non-trivial assumption induced by partial participation, which is shown to hold with high probability. Extensive experiments conducted on synthetic and real-world data demonstrate the good performance of RFedAGS.