FedCore: Straggler-Free Federated Learning with Distributed Coresets
This addresses efficiency and scalability issues for federated learning systems, particularly in scenarios with slow clients, representing a novel method for a known bottleneck.
The paper tackles the straggler problem in federated learning by introducing FedCore, a distributed coreset selection algorithm, which reduces training time by 8x while maintaining model accuracy.
Federated learning (FL) is a machine learning paradigm that allows multiple clients to collaboratively train a shared model while keeping their data on-premise. However, the straggler issue, due to slow clients, often hinders the efficiency and scalability of FL. This paper presents FedCore, an algorithm that innovatively tackles the straggler problem via the decentralized selection of coresets, representative subsets of a dataset. Contrary to existing centralized coreset methods, FedCore creates coresets directly on each client in a distributed manner, ensuring privacy preservation in FL. FedCore translates the coreset optimization problem into a more tractable k-medoids clustering problem and operates distributedly on each client. Theoretical analysis confirms FedCore's convergence, and practical evaluations demonstrate an 8x reduction in FL training time, without compromising model accuracy. Our extensive evaluations also show that FedCore generalizes well to existing FL frameworks.