Adaptive Experimentation When You Can't Experiment
This work addresses a practical issue for online services that cannot run direct A/B tests, offering an adaptive experimental design method for encouragement designs, though it is incremental as it extends pure exploration to confounded settings.
The paper tackles the problem of learning the best treatment in online services when direct experimentation is impossible due to self-selection bias, by introducing a confounded pure exploration linear bandit model and providing elimination-style algorithms with sample complexity bounds nearly matching minimax lower bounds.
This paper introduces the \emph{confounded pure exploration transductive linear bandit} (\texttt{CPET-LB}) problem. As a motivating example, often online services cannot directly assign users to specific control or treatment experiences either for business or practical reasons. In these settings, naively comparing treatment and control groups that may result from self-selection can lead to biased estimates of underlying treatment effects. Instead, online services can employ a properly randomized encouragement that incentivizes users toward a specific treatment. Our methodology provides online services with an adaptive experimental design approach for learning the best-performing treatment for such \textit{encouragement designs}. We consider a more general underlying model captured by a linear structural equation and formulate pure exploration linear bandits in this setting. Though pure exploration has been extensively studied in standard adaptive experimental design settings, we believe this is the first work considering a setting where noise is confounded. Elimination-style algorithms using experimental design methods in combination with a novel finite-time confidence interval on an instrumental variable style estimator are presented with sample complexity upper bounds nearly matching a minimax lower bound. Finally, experiments are conducted that demonstrate the efficacy of our approach.