The More is not the Merrier: Investigating the Effect of Client Size on Federated Learning
This addresses accuracy degradation in FL for applications like Mobile Edge Computing, but it is incremental as it builds on FedAvg with a specific tweak.
The paper investigates how increasing the number of clients in Federated Learning (FL) using FedAvg leads to a significant drop in learning accuracy, and proposes Knowledgeable Client Insertion (KCI) to improve accuracy by adding a few data-rich clients, resulting in faster learning gains.
Federated Learning (FL) has been introduced as a way to keep data local to clients while training a shared machine learning model, as clients train on their local data and send trained models to a central aggregator. It is expected that FL will have a huge implication on Mobile Edge Computing, the Internet of Things, and Cross-Silo FL. In this paper, we focus on the widely used FedAvg algorithm to explore the effect of the number of clients in FL. We find a significant deterioration of learning accuracy for FedAvg as the number of clients increases. To address this issue for a general application, we propose a method called Knowledgeable Client Insertion (KCI) that introduces a very small number of knowledgeable clients to the MEC setting. These knowledgeable clients are expected to have accumulated a large set of data samples to help with training. With the help of KCI, the learning accuracy of FL increases much faster even with a normal FedAvg aggregation technique. We expect this approach to be able to provide great privacy protection for clients against security attacks such as model inversion attacks. Our code is available at https://github.com/Eleanor-W/KCI_for_FL.