Statistical Consequences of Dueling Bandits
This work highlights statistical challenges in using dueling bandits for adaptive experiments, which is important for researchers and practitioners in fields like education, but it is incremental as it builds on existing bandit frameworks.
The paper investigates the statistical quality of data in dueling bandit frameworks, which are used for preference-based feedback in applications like educational interventions, finding that while these algorithms minimize cumulative regret effectively, they can inflate Type-I error rates and reduce statistical power in certain conditions.
Multi-Armed-Bandit frameworks have often been used by researchers to assess educational interventions, however, recent work has shown that it is more beneficial for a student to provide qualitative feedback through preference elicitation between different alternatives, making a dueling bandits framework more appropriate. In this paper, we explore the statistical quality of data under this framework by comparing traditional uniform sampling to a dueling bandit algorithm and find that dueling bandit algorithms perform well at cumulative regret minimisation, but lead to inflated Type-I error rates and reduced power under certain circumstances. Through these results we provide insight into the challenges and opportunities in using dueling bandit algorithms to run adaptive experiments.