Personalized Federated Learning via Variational Bayesian Inference
This addresses the problem of data scarcity and heterogeneity for federated learning clients, offering an incremental improvement over existing personalized methods.
The paper tackles model overfitting and statistical diversity in federated learning by proposing pFedBayes, a personalized method using Bayesian variational inference, which outperforms state-of-the-art algorithms by up to 11.71% on non-i.i.d. datasets.
Federated learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, this paper proposes a novel personalized federated learning method via Bayesian variational inference named pFedBayes. To alleviate the overfitting, weight uncertainty is introduced to neural networks for clients and the server. To achieve personalization, each client updates its local distribution parameters by balancing its construction error over private data and its KL divergence with global distribution from the server. Theoretical analysis gives an upper bound of averaged generalization error and illustrates that the convergence rate of the generalization error is minimax optimal up to a logarithmic factor. Experiments show that the proposed method outperforms other advanced personalized methods on personalized models, e.g., pFedBayes respectively outperforms other SOTA algorithms by 1.25%, 0.42% and 11.71% on MNIST, FMNIST and CIFAR-10 under non-i.i.d. limited data.