Fast-Convergent Federated Learning via Cyclic Aggregation
This is an incremental improvement for federated learning systems dealing with statistical and computational heterogeneity.
The paper tackles the slow convergence of federated learning under heterogeneity by proposing cyclic learning rate aggregation at the server, which reduces training iterations and improves performance without extra computational costs.
Federated learning (FL) aims at optimizing a shared global model over multiple edge devices without transmitting (private) data to the central server. While it is theoretically well-known that FL yields an optimal model -- centrally trained model assuming availability of all the edge device data at the central server -- under mild condition, in practice, it often requires massive amount of iterations until convergence, especially under presence of statistical/computational heterogeneity. This paper utilizes cyclic learning rate at the server side to reduce the number of training iterations with increased performance without any additional computational costs for both the server and the edge devices. Numerical results validate that, simply plugging-in the proposed cyclic aggregation to the existing FL algorithms effectively reduces the number of training iterations with improved performance.