Personalization Disentanglement for Federated Learning: An explainable perspective
This work addresses the challenge of model heterogeneity in federated learning for clients with diverse data, though it is incremental as it builds on existing methods.
The paper tackles the problem of balancing shared knowledge and client-specific personalization in personalized federated learning by explicitly disentangling latent representations into shared and personalized parts, resulting in models that substantially outperform vanilla methods in experiments.
Personalized federated learning (PFL) jointly trains a variety of local models through balancing between knowledge sharing across clients and model personalization per client. This paper addresses PFL via explicit disentangling latent representations into two parts to capture the shared knowledge and client-specific personalization, which leads to more reliable and effective PFL. The disentanglement is achieved by a novel Federated Dual Variational Autoencoder (FedDVA), which employs two encoders to infer the two types of representations. FedDVA can produce a better understanding of the trade-off between global knowledge sharing and local personalization in PFL. Moreover, it can be integrated with existing FL methods and turn them into personalized models for heterogeneous downstream tasks. Extensive experiments validate the advantages caused by disentanglement and show that models trained with disentangled representations substantially outperform those vanilla methods.