FedADC: Accelerated Federated Learning with Drift Control
This work is significant for researchers and practitioners in federated learning seeking to improve training speed and stability in distributed environments with heterogeneous data.
This paper addresses the challenges of accelerating federated learning (FL) and mitigating data drift in non-homogeneous distributed datasets. The authors propose FedADC, an algorithm that tackles both issues simultaneously without significant changes to the FL framework or increased computational/communication load.
Federated learning (FL) has become de facto framework for collaborative learning among edge devices with privacy concern. The core of the FL strategy is the use of stochastic gradient descent (SGD) in a distributed manner. Large scale implementation of FL brings new challenges, such as the incorporation of acceleration techniques designed for SGD into the distributed setting, and mitigation of the drift problem due to non-homogeneous distribution of local datasets. These two problems have been separately studied in the literature; whereas, in this paper, we show that it is possible to address both problems using a single strategy without any major alteration to the FL framework, or introducing additional computation and communication load. To achieve this goal, we propose FedADC, which is an accelerated FL algorithm with drift control. We empirically illustrate the advantages of FedADC.