Client-supervised Federated Learning: Towards One-model-for-all Personalization
This addresses the deployment and test-time efficiency problem for federated learning systems by enabling a one-model-for-all approach, though it appears incremental as it builds on existing personalized FL methods.
The paper tackles the challenge of model adaptation in Personalized Federated Learning by proposing a novel framework, FedCS, that learns a single robust global model to achieve competitive performance on unseen clients without requiring extra adaptation steps, with experimental results showing comparable performance to personalized methods.
Personalized Federated Learning (PerFL) is a new machine learning paradigm that delivers personalized models for diverse clients under federated learning settings. Most PerFL methods require extra learning processes on a client to adapt a globally shared model to the client-specific personalized model using its own local data. However, the model adaptation process in PerFL is still an open challenge in the stage of model deployment and test time. This work tackles the challenge by proposing a novel federated learning framework to learn only one robust global model to achieve competitive performance to those personalized models on unseen/test clients in the FL system. Specifically, we design a new Client-Supervised Federated Learning (FedCS) to unravel clients' bias on instances' latent representations so that the global model can learn both client-specific and client-agnostic knowledge. Experimental study shows that the FedCS can learn a robust FL global model for the changing data distributions of unseen/test clients. The FedCS's global model can be directly deployed to the test clients while achieving comparable performance to other personalized FL methods that require model adaptation.