Adaptive Client Selection via Q-Learning-based Whittle Index in Wireless Federated Learning
This addresses the challenge of efficient client selection in wireless Federated Learning, offering a robust solution for practical deployments, though it is incremental as it builds on existing bandit and Q-learning approaches.
The paper tackles the client selection problem in wireless Federated Learning to reduce the time needed to achieve a target learning accuracy by formulating it as a restless multi-armed bandit problem and proposing the WILF-Q method, which significantly outperforms baseline policies in learning efficiency.
We consider the client selection problem in wireless Federated Learning (FL), with the objective of reducing the total required time to achieve a certain level of learning accuracy. Since the server cannot observe the clients' dynamic states that can change their computation and communication efficiency, we formulate client selection as a restless multi-armed bandit problem. We propose a scalable and efficient approach called the Whittle Index Learning in Federated Q-learning (WILF-Q), which uses Q-learning to adaptively learn and update an approximated Whittle index associated with each client, and then selects the clients with the highest indices. Compared to existing approaches, WILF-Q does not require explicit knowledge of client state transitions or data distributions, making it well-suited for deployment in practical FL settings. Experiment results demonstrate that WILF-Q significantly outperforms existing baseline policies in terms of learning efficiency, providing a robust and efficient approach to client selection in wireless FL.