Stochastically Constrained Best Arm Identification with Thompson Sampling
This work addresses a constrained optimization problem in multi-armed bandits, which is incremental as it adapts an existing method to a new scenario.
The paper tackles the problem of best arm identification with stochastic constraints by extending Thompson sampling to this setting, establishing asymptotic optimality and demonstrating superior performance in numerical examples.
We consider the problem of the best arm identification in the presence of stochastic constraints, where there is a finite number of arms associated with multiple performance measures. The goal is to identify the arm that optimizes the objective measure subject to constraints on the remaining measures. We will explore the popular idea of Thompson sampling (TS) as a means to solve it. To the best of our knowledge, it is the first attempt to extend TS to this problem. We will design a TS-based sampling algorithm, establish its asymptotic optimality in the rate of posterior convergence, and demonstrate its superior performance using numerical examples.