Adaptive Experimental Design for Policy Learning
This work addresses the problem of efficient policy learning in adaptive experiments for decision-makers, representing an incremental improvement with a novel method for a known bottleneck in contextual bandits.
The study tackles the contextual best arm identification problem by designing an adaptive experiment to identify the best treatment arm based on covariates, proposing the Adaptive Sampling-Policy Learning (PLAS) strategy, which is proven to be minimax rate-optimal with its regret upper bound matching the derived lower bound as the number of experimental units increases.
This study investigates the contextual best arm identification (BAI) problem, aiming to design an adaptive experiment to identify the best treatment arm conditioned on contextual information (covariates). We consider a decision-maker who assigns treatment arms to experimental units during an experiment and recommends the estimated best treatment arm based on the contexts at the end of the experiment. The decision-maker uses a policy for recommendations, which is a function that provides the estimated best treatment arm given the contexts. In our evaluation, we focus on the worst-case expected regret, a relative measure between the expected outcomes of an optimal policy and our proposed policy. We derive a lower bound for the expected simple regret and then propose a strategy called Adaptive Sampling-Policy Learning (PLAS). We prove that this strategy is minimax rate-optimal in the sense that its leading factor in the regret upper bound matches the lower bound as the number of experimental units increases.