FedCAT: Towards Accurate Federated Learning via Device Concatenation
This addresses the challenge of data heterogeneity in federated learning for privacy-preserving distributed machine learning, representing an incremental improvement over existing methods.
The paper tackles the problem of reduced classification accuracy in federated learning under non-IID data scenarios by introducing FedCat, which uses device concatenation to alleviate weight divergence, achieving significant accuracy improvements over state-of-the-art methods without extra communication overhead.
As a promising distributed machine learning paradigm, Federated Learning (FL) enables all the involved devices to train a global model collaboratively without exposing their local data privacy. However, for non-IID scenarios, the classification accuracy of FL models decreases drastically due to the weight divergence caused by data heterogeneity. Although various FL variants have been studied to improve model accuracy, most of them still suffer from the problem of non-negligible communication and computation overhead. In this paper, we introduce a novel FL approach named Fed-Cat that can achieve high model accuracy based on our proposed device selection strategy and device concatenation-based local training method. Unlike conventional FL methods that aggregate local models trained on individual devices, FedCat periodically aggregates local models after their traversals through a series of logically concatenated devices, which can effectively alleviate the model weight divergence problem. Comprehensive experimental results on four well-known benchmarks show that our approach can significantly improve the model accuracy of state-of-the-art FL methods without causing extra communication overhead.