Personalized Federated Learning with Multiple Known Clusters
This work addresses the problem of improving model personalization in federated learning for users with clustered structures, offering an incremental method over existing approaches.
The paper tackles personalized federated learning by leveraging known user clusters, developing an algorithm that regularizes parameters within and between clusters to improve model performance. It demonstrates theoretical and empirical advantages over independent and shared learning, showing gains in simulated and real-world data.
We consider the problem of personalized federated learning when there are known cluster structures within users. An intuitive approach would be to regularize the parameters so that users in the same cluster share similar model weights. The distances between the clusters can then be regularized to reflect the similarity between different clusters of users. We develop an algorithm that allows each cluster to communicate independently and derive the convergence results. We study a hierarchical linear model to theoretically demonstrate that our approach outperforms agents learning independently and agents learning a single shared weight. Finally, we demonstrate the advantages of our approach using both simulated and real-world data.