FAVANO: Federated AVeraging with Asynchronous NOdes
This work addresses the challenge of deploying federated learning in practical, resource-constrained settings with heterogeneous clients, representing an incremental improvement over existing methods.
The paper tackles the problem of scaling federated learning in resource-constrained environments by proposing FAVANO, a novel asynchronous framework that addresses bias from varying client speeds, and it shows experimental outperformance over current methods on standard benchmarks.
In this paper, we propose a novel centralized Asynchronous Federated Learning (FL) framework, FAVANO, for training Deep Neural Networks (DNNs) in resource-constrained environments. Despite its popularity, ``classical'' federated learning faces the increasingly difficult task of scaling synchronous communication over large wireless networks. Moreover, clients typically have different computing resources and therefore computing speed, which can lead to a significant bias (in favor of ``fast'' clients) when the updates are asynchronous. Therefore, practical deployment of FL requires to handle users with strongly varying computing speed in communication/resource constrained setting. We provide convergence guarantees for FAVANO in a smooth, non-convex environment and carefully compare the obtained convergence guarantees with existing bounds, when they are available. Experimental results show that the FAVANO algorithm outperforms current methods on standard benchmarks.