Federated Learning for Non-IID Data via Client Variance Reduction and Adaptive Server Update
This addresses a fundamental bottleneck in federated learning for decentralized data scenarios, though it appears incremental as it builds on existing methods.
The paper tackles the problem of Non-IID data distribution in federated learning, which slows convergence and degrades performance, by proposing ComFed, a method that improves state-of-the-art algorithms on the Cifar-10 classification task.
Federated learning (FL) is an emerging technique used to collaboratively train a global machine learning model while keeping the data localized on the user devices. The main obstacle to FL's practical implementation is the Non-Independent and Identical (Non-IID) data distribution across users, which slows convergence and degrades performance. To tackle this fundamental issue, we propose a method (ComFed) that enhances the whole training process on both the client and server sides. The key idea of ComFed is to simultaneously utilize client-variance reduction techniques to facilitate server aggregation and global adaptive update techniques to accelerate learning. Our experiments on the Cifar-10 classification task show that ComFed can improve state-of-the-art algorithms dedicated to Non-IID data.