A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning
This work addresses efficiency challenges in federated learning for heterogeneous and low-bandwidth environments, representing an incremental improvement by combining existing techniques.
The paper tackles the problem of stale model updates in Asynchronous Federated Learning (AFL) due to device heterogeneity and low bandwidth by jointly optimizing local updating and gradient compression, resulting in a 56% reduction in communication consumption and 55% reduction in training time while maintaining competitive performance.
Asynchronous Federated Learning (AFL) confronts inherent challenges arising from the heterogeneity of devices (e.g., their computation capacities) and low-bandwidth environments, both potentially causing stale model updates (e.g., local gradients) for global aggregation. Traditional approaches mitigating the staleness of updates typically focus on either adjusting the local updating or gradient compression, but not both. Recognizing this gap, we introduce a novel approach that synergizes local updating with gradient compression. Our research begins by examining the interplay between local updating frequency and gradient compression rate, and their collective impact on convergence speed. The theoretical upper bound shows that the local updating frequency and gradient compression rate of each device are jointly determined by its computing power, communication capabilities and other factors. Building on this foundation, we propose an AFL framework called FedLuck that adaptively optimizes both local update frequency and gradient compression rates. Experiments on image classification and speech recognization show that FedLuck reduces communication consumption by 56% and training time by 55% on average, achieving competitive performance in heterogeneous and low-bandwidth scenarios compared to the baselines.