Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning
This addresses the challenge of adapting federated learning to non-stationary data streams for clients, but it is incremental as it builds on existing personalized federated learning methods.
The paper tackles the problem of federated learning in dynamic environments where clients need real-time predictions on streaming data, proposing a personalized algorithm that combines locally fine-tuned models with federated models, and experiments on real datasets show its effectiveness.
Federated learning is renowned for its efficacy in distributed model training, ensuring that users, called clients, retain data privacy by not disclosing their data to the central server that orchestrates collaborations. Most previous work on federated learning assumes that clients possess static batches of training data. However, clients may also need to make real-time predictions on streaming data in non-stationary environments. In such dynamic environments, employing pre-trained models may be inefficient, as they struggle to adapt to the constantly evolving data streams. To address this challenge, clients can fine-tune models online, leveraging their observed data to enhance performance. Despite the potential benefits of client participation in federated online model fine-tuning, existing analyses have not conclusively demonstrated its superiority over local model fine-tuning. To bridge this gap, the present paper develops a novel personalized federated learning algorithm, wherein each client constructs a personalized model by combining a locally fine-tuned model with multiple federated models learned by the server over time. Theoretical analysis and experiments on real datasets corroborate the effectiveness of this approach for real-time predictions and federated model fine-tuning.