Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learning
This addresses communication bottlenecks in Federated Learning for real-world applications, offering an incremental improvement over existing methods.
The paper tackles the high communication cost in Federated Learning by proposing FedLUAR, a method that recycles previous updates instead of discarding them, reducing communication cost to 17% while maintaining similar accuracy on AG News compared to FedAvg.
Expensive communication cost is a common performance bottleneck in Federated Learning (FL), which makes it less appealing in real-world applications. Many communication-efficient FL methods focus on discarding a part of model updates mostly based on gradient magnitude. In this study, we find that recycling previous updates, rather than simply dropping them, more effectively reduces the communication cost while maintaining FL performance. We propose FedLUAR, a Layer-wise Update Aggregation with Recycling scheme for communication-efficient FL. We first define a useful metric that quantifies the extent to which the aggregated gradients influences the model parameter values in each layer. FedLUAR selects a few layers based on the metric and recycles their previous updates on the server side. Our extensive empirical study demonstrates that the update recycling scheme significantly reduces the communication cost while maintaining model accuracy. For example, our method achieves nearly the same AG News accuracy as FedAvg, while reducing the communication cost to just 17%.