Optimizing the Collaboration Structure in Cross-Silo Federated Learning
This addresses the challenge of data heterogeneity in federated learning for applications like healthcare or finance, though it is incremental as it builds on existing clustered FL methods.
The paper tackles the negative transfer problem in federated learning, where global models can underperform local ones, by proposing FedCollab, a framework that clusters clients into coalitions based on distribution similarity and data quantity, resulting in improved performance across various datasets and algorithms.
In federated learning (FL), multiple clients collaborate to train machine learning models together while keeping their data decentralized. Through utilizing more training data, FL suffers from the potential negative transfer problem: the global FL model may even perform worse than the models trained with local data only. In this paper, we propose FedCollab, a novel FL framework that alleviates negative transfer by clustering clients into non-overlapping coalitions based on their distribution distances and data quantities. As a result, each client only collaborates with the clients having similar data distributions, and tends to collaborate with more clients when it has less data. We evaluate our framework with a variety of datasets, models, and types of non-IIDness. Our results demonstrate that FedCollab effectively mitigates negative transfer across a wide range of FL algorithms and consistently outperforms other clustered FL algorithms.