Gradient Coreset for Federated Learning
This work addresses efficiency and noise robustness for FL systems, particularly on resource-constrained edge devices, but is incremental as it builds on existing coreset methods.
The paper tackles the problem of improving efficiency and robustness in Federated Learning (FL) by proposing a gradient-based coreset selection algorithm (GCFL) that selects subsets of data at clients every K rounds, demonstrating it is more compute and energy efficient than FL, robust to noise, and achieves significant performance gains, with experiments on four real-world datasets showing these benefits.
Federated Learning (FL) is used to learn machine learning models with data that is partitioned across multiple clients, including resource-constrained edge devices. It is therefore important to devise solutions that are efficient in terms of compute, communication, and energy consumption, while ensuring compliance with the FL framework's privacy requirements. Conventional approaches to these problems select a weighted subset of the training dataset, known as coreset, and learn by fitting models on it. Such coreset selection approaches are also known to be robust to data noise. However, these approaches rely on the overall statistics of the training data and are not easily extendable to the FL setup. In this paper, we propose an algorithm called Gradient based Coreset for Robust and Efficient Federated Learning (GCFL) that selects a coreset at each client, only every $K$ communication rounds and derives updates only from it, assuming the availability of a small validation dataset at the server. We demonstrate that our coreset selection technique is highly effective in accounting for noise in clients' data. We conduct experiments using four real-world datasets and show that GCFL is (1) more compute and energy efficient than FL, (2) robust to various kinds of noise in both the feature space and labels, (3) preserves the privacy of the validation dataset, and (4) introduces a small communication overhead but achieves significant gains in performance, particularly in cases when the clients' data is noisy.