Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning
This work addresses communication efficiency for federated learning systems, offering a novel integration that improves performance, though it is incremental in nature.
The paper tackles the problem of high communication costs in federated learning by introducing Scafflix, an algorithm that combines explicit personalization with local training, achieving doubly accelerated communication as demonstrated theoretically and practically.
Federated Learning is an evolving machine learning paradigm, in which multiple clients perform computations based on their individual private data, interspersed by communication with a remote server. A common strategy to curtail communication costs is Local Training, which consists in performing multiple local stochastic gradient descent steps between successive communication rounds. However, the conventional approach to local training overlooks the practical necessity for client-specific personalization, a technique to tailor local models to individual needs. We introduce Scafflix, a novel algorithm that efficiently integrates explicit personalization with local training. This innovative approach benefits from these two techniques, thereby achieving doubly accelerated communication, as we demonstrate both in theory and practice.