Policy Evaluation with Latent Confounders via Optimal Balance
This addresses a critical problem in fields like medicine where exploration is costly, though it is an incremental improvement over existing methods by handling latent confounders via proxies.
The paper tackles policy evaluation in contextual bandits with latent confounders by developing an importance weighting method that avoids fitting latent outcome models, and it demonstrates empirical effectiveness when confounders are latent.
Evaluating novel contextual bandit policies using logged data is crucial in applications where exploration is costly, such as medicine. But it usually relies on the assumption of no unobserved confounders, which is bound to fail in practice. We study the question of policy evaluation when we instead have proxies for the latent confounders and develop an importance weighting method that avoids fitting a latent outcome regression model. We show that unlike the unconfounded case no single set of weights can give unbiased evaluation for all outcome models, yet we propose a new algorithm that can still provably guarantee consistency by instead minimizing an adversarial balance objective. We further develop tractable algorithms for optimizing this objective and demonstrate empirically the power of our method when confounders are latent.