Guaranteed Fixed-Confidence Best Arm Identification in Multi-Armed Bandits: Simple Sequential Elimination Algorithms
This addresses the fixed-confidence best arm identification problem for adaptive sampling in multi-armed bandits, offering incremental improvements with efficient algorithms.
The paper tackles the problem of identifying the best arm in multi-armed bandits with fixed confidence, proposing simple sequential elimination algorithms like VT and PW variants, and shows they guarantee optimal strategies under Bayesian priors and compare favorably with state-of-the-art methods in numerical results.
We consider the problem of finding, through adaptive sampling, which of $n$ options (arms) has the largest mean. Our objective is to determine a rule which identifies the best arm with a fixed minimum confidence using as few observations as possible, i.e. this is a fixed-confidence (FC) best arm identification (BAI) in multi-armed bandits. We study such problems under the Bayesian setting with both Bernoulli and Gaussian arms. We propose to use the classical "vector at a time" (VT) rule, which samples each remaining arm once in each round. We show how VT can be implemented and analyzed in our Bayesian setting and be improved by early elimination. Our analysis show that these algorithms guarantee an optimal strategy under the prior. We also propose and analyze a variant of the classical "play the winner" (PW) algorithm. Numerical results show that these rules compare favorably with state-of-art algorithms.