Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification
This work addresses a practical problem for practitioners conducting adaptive experiments by unifying previously separate approaches, though it is incremental in building on existing algorithms and theories.
The paper tackles the competing priorities in adaptive experiments—maximizing welfare during the experiment and quickly concluding it for population-wide treatment—by proposing a unified model that bridges regret minimization and best-arm identification. It shows that adjusting a single parameter in algorithms like top-two Thompson sampling can optimize a broad class of objectives, often achieving substantial reductions in experiment duration with minimal impact on regret.
Practitioners conducting adaptive experiments often encounter two competing priorities: maximizing total welfare (or `reward') through effective treatment assignment and swiftly concluding experiments to implement population-wide treatments. Current literature addresses these priorities separately, with regret minimization studies focusing on the former and best-arm identification research on the latter. This paper bridges this divide by proposing a unified model that simultaneously accounts for within-experiment performance and post-experiment outcomes. We provide a sharp theory of optimal performance in large populations that not only unifies canonical results in the literature but also uncovers novel insights. Our theory reveals that familiar algorithms, such as the recently proposed top-two Thompson sampling algorithm, can optimize a broad class of objectives if a single scalar parameter is appropriately adjusted. In addition, we demonstrate that substantial reductions in experiment duration can often be achieved with minimal impact on both within-experiment and post-experiment regret.