Good Arm Identification via Bandit Feedback
This addresses a pure-exploration challenge in bandit problems, such as clinical trials, but is incremental as it adapts existing bandit frameworks to a specific identification task.
The paper tackles the problem of good arm identification in stochastic multi-armed bandits, where the goal is to find arms with expected reward above a threshold, and it derives a tight lower bound on sample complexity and proposes an algorithm that nearly matches this bound, with experimental validation showing outperformance over naive methods.
We consider a novel stochastic multi-armed bandit problem called {\em good arm identification} (GAI), where a good arm is defined as an arm with expected reward greater than or equal to a given threshold. GAI is a pure-exploration problem that a single agent repeats a process of outputting an arm as soon as it is identified as a good one before confirming the other arms are actually not good. The objective of GAI is to minimize the number of samples for each process. We find that GAI faces a new kind of dilemma, the {\em exploration-exploitation dilemma of confidence}, which is different difficulty from the best arm identification. As a result, an efficient design of algorithms for GAI is quite different from that for the best arm identification. We derive a lower bound on the sample complexity of GAI that is tight up to the logarithmic factor $\mathrm{O}(\log \frac{1}δ)$ for acceptance error rate $δ$. We also develop an algorithm whose sample complexity almost matches the lower bound. We also confirm experimentally that our proposed algorithm outperforms naive algorithms in synthetic settings based on a conventional bandit problem and clinical trial researches for rheumatoid arthritis.