FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy
This work addresses statistical heterogeneity in personalized federated learning for applications like hospitals and mobile devices, offering a sample-specific approach that is incremental over existing methods.
The paper tackles the problem of separating global and personalized information in federated learning by proposing FedCP, which uses conditional policies for each sample to process features separately, achieving up to 6.69% improvement over state-of-the-art methods in experiments across computer vision and natural language processing domains.
Recently, personalized federated learning (pFL) has attracted increasing attention in privacy protection, collaborative learning, and tackling statistical heterogeneity among clients, e.g., hospitals, mobile smartphones, etc. Most existing pFL methods focus on exploiting the global information and personalized information in the client-level model parameters while neglecting that data is the source of these two kinds of information. To address this, we propose the Federated Conditional Policy (FedCP) method, which generates a conditional policy for each sample to separate the global information and personalized information in its features and then processes them by a global head and a personalized head, respectively. FedCP is more fine-grained to consider personalization in a sample-specific manner than existing pFL methods. Extensive experiments in computer vision and natural language processing domains show that FedCP outperforms eleven state-of-the-art methods by up to 6.69%. Furthermore, FedCP maintains its superiority when some clients accidentally drop out, which frequently happens in mobile settings. Our code is public at https://github.com/TsingZ0/FedCP.