Personalized Federated Learning via Amortized Bayesian Meta-Learning
This work addresses the problem of improving model performance for individual clients in federated learning, offering a novel method for personalization.
The paper tackles the challenge of statistical heterogeneity in federated learning by proposing FedABML, a personalized federated learning algorithm based on amortized Bayesian meta-learning, which outperforms competitive baselines in empirical results.
Federated learning is a decentralized and privacy-preserving technique that enables multiple clients to collaborate with a server to learn a global model without exposing their private data. However, the presence of statistical heterogeneity among clients poses a challenge, as the global model may struggle to perform well on each client's specific task. To address this issue, we introduce a new perspective on personalized federated learning through Amortized Bayesian Meta-Learning. Specifically, we propose a novel algorithm called \emph{FedABML}, which employs hierarchical variational inference across clients. The global prior aims to capture representations of common intrinsic structures from heterogeneous clients, which can then be transferred to their respective tasks and aid in the generation of accurate client-specific approximate posteriors through a few local updates. Our theoretical analysis provides an upper bound on the average generalization error and guarantees the generalization performance on unseen data. Finally, several empirical results are implemented to demonstrate that \emph{FedABML} outperforms several competitive baselines.