Achieving the Pareto Frontier of Regret Minimization and Best Arm Identification in Multi-Armed Bandits
This work addresses a fundamental trade-off in multi-armed bandits, which is incremental but provides precise constants and practical improvements for applications such as recommendation systems and drug discovery.
The paper tackles the trade-off between regret minimization and best arm identification in multi-armed bandits, showing that no algorithm can be optimal for both objectives simultaneously, and proposes BoBW-lil'UCB(γ) which achieves order-wise optimal performance for each objective under different γ values, with demonstrated improvements in time complexity and regret on datasets like MovieLens and Published Kinase Inhibitor Set.
We study the Pareto frontier of two archetypal objectives in multi-armed bandits, namely, regret minimization (RM) and best arm identification (BAI) with a fixed horizon. It is folklore that the balance between exploitation and exploration is crucial for both RM and BAI, but exploration is more critical in achieving the optimal performance for the latter objective. To this end, we design and analyze the BoBW-lil'UCB$(γ)$ algorithm. Complementarily, by establishing lower bounds on the regret achievable by any algorithm with a given BAI failure probability, we show that (i) no algorithm can simultaneously perform optimally for both the RM and BAI objectives, and (ii) BoBW-lil'UCB$(γ)$ achieves order-wise optimal performance for RM or BAI under different values of $γ$. Our work elucidates the trade-off more precisely by showing how the constants in previous works depend on certain hardness parameters. Finally, we show that BoBW-lil'UCB outperforms a close competitor UCB$_α$ (Degenne et al., 2019) in terms of the time complexity and the regret on diverse datasets such as MovieLens and Published Kinase Inhibitor Set.