Joint AP Probing and Scheduling: A Contextual Bandit Approach
This addresses efficient resource allocation in mobile networks under uncertainty, offering a novel extension to contextual bandits for sequential decision-making problems.
The paper tackles the problem of scheduling wireless access points (APs) with unknown data rates by allowing joint probing and selection, modeling it as a contextual bandit with probing (CBwP). They present an efficient algorithm and establish regret bounds for Bernoulli data rates.
We consider a set of APs with unknown data rates that cooperatively serve a mobile client. The data rate of each link is i.i.d. sampled from a distribution that is unknown a priori. In contrast to traditional link scheduling problems under uncertainty, we assume that in each time step, the device can probe a subset of links before deciding which one to use. We model this problem as a contextual bandit problem with probing (CBwP) and present an efficient algorithm. We further establish the regret of our algorithm for links with Bernoulli data rates. Our CBwP model is a novel extension of the classic contextual bandit model and can potentially be applied to a large class of sequential decision-making problems that involve joint probing and play under uncertainty.