FedGCS: A Generative Framework for Efficient Client Selection in Federated Learning via Gradient-based Optimization
This addresses the problem of statistical and system heterogeneity and high energy consumption in Federated Learning for practitioners, but it is incremental as it builds on existing client selection methods with a novel generative approach.
FedGCS tackles the challenge of efficient client selection in Federated Learning by proposing a generative framework that recasts selection as a generative task, achieving improvements in model performance, latency, and energy consumption through gradient-based optimization.
Federated Learning faces significant challenges in statistical and system heterogeneity, along with high energy consumption, necessitating efficient client selection strategies. Traditional approaches, including heuristic and learning-based methods, fall short of addressing these complexities holistically. In response, we propose FedGCS, a novel generative client selection framework that innovatively recasts the client selection process as a generative task. Drawing inspiration from the methodologies used in large language models, FedGCS efficiently encodes abundant decision-making knowledge within a continuous representation space, enabling efficient gradient-based optimization to search for optimal client selection that will be finally output via generation. The framework comprises four steps: (1) automatic collection of diverse "selection-score" pair data using classical client selection methods; (2) training an encoder-evaluator-decoder framework on this data to construct a continuous representation space; (3) employing gradient-based optimization in this space for optimal client selection; (4) generating the final optimal client selection via using beam search for the well-trained decoder. FedGCS outperforms traditional methods by being more comprehensive, generalizable, and efficient, simultaneously optimizing for model performance, latency, and energy consumption. The effectiveness of FedGCS is proven through extensive experimental analyses.