PeFLL: Personalized Federated Learning by Learning to Learn
This addresses the challenge of efficient and scalable personalized model training in federated learning systems, particularly for low-data scenarios and new clients, with incremental improvements over prior methods.
The paper tackles the problem of personalized federated learning by introducing PeFLL, which improves model accuracy, especially with limited data, and reduces computational and communication costs for both existing and future clients, achieving state-of-the-art performance in benchmarks.
We present PeFLL, a new personalized federated learning algorithm that improves over the state-of-the-art in three aspects: 1) it produces more accurate models, especially in the low-data regime, and not only for clients present during its training phase, but also for any that may emerge in the future; 2) it reduces the amount of on-client computation and client-server communication by providing future clients with ready-to-use personalized models that require no additional finetuning or optimization; 3) it comes with theoretical guarantees that establish generalization from the observed clients to future ones. At the core of PeFLL lies a learning-to-learn approach that jointly trains an embedding network and a hypernetwork. The embedding network is used to represent clients in a latent descriptor space in a way that reflects their similarity to each other. The hypernetwork takes as input such descriptors and outputs the parameters of fully personalized client models. In combination, both networks constitute a learning algorithm that achieves state-of-the-art performance in several personalized federated learning benchmarks.