Boosting Resource-Constrained Federated Learning Systems with Guessed Updates
This work addresses performance issues in federated learning systems for edge computing, offering an incremental improvement by enhancing existing algorithms.
The paper tackles the problem of slow convergence in federated learning due to resource constraints and systems heterogeneity by proposing GEL, a guess and learn algorithm that enables edge devices to perform additional gradientless updates, boosting empirical convergence by up to 40% and reducing the need for exhaustive learning rate tuning.
Federated learning (FL) enables a set of client devices to collaboratively train a model without sharing raw data. This process, though, operates under the constrained computation and communication resources of edge devices. These constraints combined with systems heterogeneity force some participating clients to perform fewer local updates than expected by the server, thus slowing down convergence. Exhaustive tuning of hyperparameters in FL, furthermore, can be resource-intensive, without which the convergence is adversely affected. In this work, we propose GEL, the guess and learn algorithm. GEL enables constrained edge devices to perform additional learning through guessed updates on top of gradient-based steps. These guesses are gradientless, i.e., participating clients leverage them for free. Our generic guessing algorithm (i) can be flexibly combined with several state-of-the-art algorithms including FEDPROX, FEDNOVA, FEDYOGI or SCALEFL; and (ii) achieves significantly improved performance when the learning rates are not best tuned. We conduct extensive experiments and show that GEL can boost empirical convergence by up to 40% in resource constrained networks while relieving the need for exhaustive learning rate tuning.