A Unified Framework of Policy Learning for Contextual Bandit with Confounding Bias and Missing Observations
This work addresses data deficiencies in offline contextual bandits, which is incremental as it builds on existing methods to handle specific biases.
The paper tackles the offline contextual bandit problem by addressing unobserved confounders and missing observations in data, proposing the Causal-Adjusted Pessimistic (CAP) algorithm to learn an optimal policy with a theoretical upper bound on suboptimality.
We study the offline contextual bandit problem, where we aim to acquire an optimal policy using observational data. However, this data usually contains two deficiencies: (i) some variables that confound actions are not observed, and (ii) missing observations exist in the collected data. Unobserved confounders lead to a confounding bias and missing observations cause bias and inefficiency problems. To overcome these challenges and learn the optimal policy from the observed dataset, we present a new algorithm called Causal-Adjusted Pessimistic (CAP) policy learning, which forms the reward function as the solution of an integral equation system, builds a confidence set, and greedily takes action with pessimism. With mild assumptions on the data, we develop an upper bound to the suboptimality of CAP for the offline contextual bandit problem.