Self-Aware Personalized Federated Learning
This work addresses the problem of optimizing personalization in federated learning for clients with non-aligned objectives, representing an incremental improvement over existing methods.
The paper tackles the challenge of balancing local and global model training in personalized federated learning by developing a self-aware method that uses uncertainty quantification to adjust training steps and aggregation, achieving significantly improved personalization performance on datasets like MNIST, CIFAR10, and Amazon Alexa audio data.
In the context of personalized federated learning (FL), the critical challenge is to balance local model improvement and global model tuning when the personal and global objectives may not be exactly aligned. Inspired by Bayesian hierarchical models, we develop a self-aware personalized FL method where each client can automatically balance the training of its local personal model and the global model that implicitly contributes to other clients' training. Such a balance is derived from the inter-client and intra-client uncertainty quantification. A larger inter-client variation implies more personalization is needed. Correspondingly, our method uses uncertainty-driven local training steps and aggregation rule instead of conventional local fine-tuning and sample size-based aggregation. With experimental studies on synthetic data, Amazon Alexa audio data, and public datasets such as MNIST, FEMNIST, CIFAR10, and Sent140, we show that our proposed method can achieve significantly improved personalization performance compared with the existing counterparts.