Communication-Efficient Federated Learning with Adaptive Compression under Dynamic Bandwidth
This work addresses communication inefficiencies in federated learning for distributed systems, but it is incremental as it builds on prior compression techniques by adding adaptive mechanisms.
The paper tackles the problem of high communication overhead in federated learning due to dynamic and heterogeneous client bandwidths by proposing AdapComFL, an algorithm that adaptively compresses local models based on predicted bandwidth, achieving more efficient communication with competitive accuracy compared to existing methods.
Federated learning can train models without directly providing local data to the server. However, the frequent updating of the local model brings the problem of large communication overhead. Recently, scholars have achieved the communication efficiency of federated learning mainly by model compression. But they ignore two problems: 1) network state of each client changes dynamically; 2) network state among clients is not the same. The clients with poor bandwidth update local model slowly, which leads to low efficiency. To address this challenge, we propose a communication-efficient federated learning algorithm with adaptive compression under dynamic bandwidth (called AdapComFL). Concretely, each client performs bandwidth awareness and bandwidth prediction. Then, each client adaptively compresses its local model via the improved sketch mechanism based on his predicted bandwidth. Further, the server aggregates sketched models with different sizes received. To verify the effectiveness of the proposed method, the experiments are based on real bandwidth data which are collected from the network topology we build, and benchmark datasets which are obtained from open repositories. We show the performance of AdapComFL algorithm, and compare it with existing algorithms. The experimental results show that our AdapComFL achieves more efficient communication as well as competitive accuracy compared to existing algorithms.