Find Your Friends: Personalized Federated Learning with the Right Collaborators
This addresses the challenge of personalized model training in federated settings for clients with heterogeneous data, representing an incremental advancement in decentralized collaboration methods.
The paper tackles the problem of data heterogeneity and lack of a trusted central party in federated learning by introducing FedeRiCo, a decentralized framework where clients select collaborators based on estimated model utilities, resulting in consistent performance improvements over local-only training on benchmark datasets.
In the traditional federated learning setting, a central server coordinates a network of clients to train one global model. However, the global model may serve many clients poorly due to data heterogeneity. Moreover, there may not exist a trusted central party that can coordinate the clients to ensure that each of them can benefit from others. To address these concerns, we present a novel decentralized framework, FedeRiCo, where each client can learn as much or as little from other clients as is optimal for its local data distribution. Based on expectation-maximization, FedeRiCo estimates the utilities of other participants' models on each client's data so that everyone can select the right collaborators for learning. As a result, our algorithm outperforms other federated, personalized, and/or decentralized approaches on several benchmark datasets, being the only approach that consistently performs better than training with local data only.