Stochastic bandits with arm-dependent delays
This work addresses delays in bandit applications where existing methods rely on strong assumptions, making it relevant for scenarios with varying or heavy-tailed delays, though it is incremental in weakening these assumptions.
The paper tackled the problem of stochastic bandits with arm-dependent and heavy-tailed delays by proposing a UCB-based algorithm called PatientBandits, which achieves both problem-dependent and problem-independent regret bounds and provides performance lower bounds.
Significant work has been recently dedicated to the stochastic delayed bandit setting because of its relevance in applications. The applicability of existing algorithms is however restricted by the fact that strong assumptions are often made on the delay distributions, such as full observability, restrictive shape constraints, or uniformity over arms. In this work, we weaken them significantly and only assume that there is a bound on the tail of the delay. In particular, we cover the important case where the delay distributions vary across arms, and the case where the delays are heavy-tailed. Addressing these difficulties, we propose a simple but efficient UCB-based algorithm called the PatientBandits. We provide both problems-dependent and problems-independent bounds on the regret as well as performance lower bounds.