Communication-Efficient Federated Learning with Adaptive Number of Participants
This addresses communication bottlenecks in federated learning for decentralized training applications, but it is incremental as it builds on existing client selection strategies.
The paper tackles the problem of communication inefficiency in federated learning under heterogeneous and dynamic client participation by introducing an adaptive mechanism to dynamically determine the optimal number of clients per round, achieving up to 30% communication savings without compromising model accuracy.
Rapid scaling of deep learning models has enabled performance gains across domains, yet it introduced several challenges. Federated Learning (FL) has emerged as a promising framework to address these concerns by enabling decentralized training. Nevertheless, communication efficiency remains a key bottleneck in FL, particularly under heterogeneous and dynamic client participation. Existing methods, such as FedAvg and FedProx, or other approaches, including client selection strategies, attempt to mitigate communication costs. However, the problem of choosing the number of clients in a training round remains extremely underexplored. We introduce Intelligent Selection of Participants (ISP), an adaptive mechanism that dynamically determines the optimal number of clients per round to enhance communication efficiency without compromising model accuracy. We validate the effectiveness of ISP across diverse setups, including vision transformers, real-world ECG classification, and training with gradient compression. Our results show consistent communication savings of up to 30\% without losing the final quality. Applying ISP to different real-world ECG classification setups highlighted the selection of the number of clients as a separate task of federated learning.