Aggregation Delayed Federated Learning
This addresses the challenge of data heterogeneity for federated learning systems, but it appears incremental as it modifies the aggregation timing rather than introducing a new paradigm.
The paper tackles performance reduction in federated learning on non-IID data by introducing redistribution rounds that delay aggregation, showing significant improvements in experiments across multiple tasks.
Federated learning is a distributed machine learning paradigm where multiple data owners (clients) collaboratively train one machine learning model while keeping data on their own devices. The heterogeneity of client datasets is one of the most important challenges of federated learning algorithms. Studies have found performance reduction with standard federated algorithms, such as FedAvg, on non-IID data. Many existing works on handling non-IID data adopt the same aggregation framework as FedAvg and focus on improving model updates either on the server side or on clients. In this work, we tackle this challenge in a different view by introducing redistribution rounds that delay the aggregation. We perform experiments on multiple tasks and show that the proposed framework significantly improves the performance on non-IID data.