WarmFed: Federated Learning with Warm-Start for Globalization and Personalization Via Personalized Diffusion Models
This work addresses the trade-off between globalization and personalization in federated learning, offering a method to enhance both aspects for clients in distributed settings, though it appears incremental as it builds on existing FL and personalization techniques.
The paper tackles the dilemma in federated learning between global and personalized models by introducing WarmFed, which uses personalized diffusion models for warm-start initialization to simultaneously improve both model types, achieving substantial gains in just one-shot and five communication rounds.
Federated Learning (FL) stands as a prominent distributed learning paradigm among multiple clients to achieve a unified global model without privacy leakage. In contrast to FL, Personalized federated learning aims at serving for each client in achieving persoanlized model. However, previous FL frameworks have grappled with a dilemma: the choice between developing a singular global model at the server to bolster globalization or nurturing personalized model at the client to accommodate personalization. Instead of making trade-offs, this paper commences its discourse from the pre-trained initialization, obtaining resilient global information and facilitating the development of both global and personalized models. Specifically, we propose a novel method called WarmFed to achieve this. WarmFed customizes Warm-start through personalized diffusion models, which are generated by local efficient fine-tunining (LoRA). Building upon the Warm-Start, we advance a server-side fine-tuning strategy to derive the global model, and propose a dynamic self-distillation (DSD) to procure more resilient personalized models simultaneously. Comprehensive experiments underscore the substantial gains of our approach across both global and personalized models, achieved within just one-shot and five communication(s).