FedCM: Federated Learning with Client-level Momentum
This addresses challenges in real-world federated learning applications, but it is incremental as it builds on existing methods like Federated Averaging.
The paper tackles the problems of partial participation and client heterogeneity in federated learning by proposing FedCM, which aggregates global gradient information with a momentum-like term, resulting in improved stability and superior performance across various tasks.
Federated Learning is a distributed machine learning approach which enables model training without data sharing. In this paper, we propose a new federated learning algorithm, Federated Averaging with Client-level Momentum (FedCM), to tackle problems of partial participation and client heterogeneity in real-world federated learning applications. FedCM aggregates global gradient information in previous communication rounds and modifies client gradient descent with a momentum-like term, which can effectively correct the bias and improve the stability of local SGD. We provide theoretical analysis to highlight the benefits of FedCM. We also perform extensive empirical studies and demonstrate that FedCM achieves superior performance in various tasks and is robust to different levels of client numbers, participation rate and client heterogeneity.