The Adaptive Doubly Robust Estimator for Policy Evaluation in Adaptive Experiments and a Paradox Concerning Logging Policy
This work addresses causal inference challenges in adaptive experiments, offering a novel estimator with potential applications in fields like online advertising or clinical trials, though it appears incremental as it builds on existing doubly robust methods.
The paper tackles the problem of policy evaluation in adaptive experiments by proposing an adaptive doubly robust estimator that works with dependent samples, achieving asymptotic normality through adaptive-fitting. It also reports an empirical paradox where the proposed estimator outperforms others using the true logging policy, confirmed via simulations.
The doubly robust (DR) estimator, which consists of two nuisance parameters, the conditional mean outcome and the logging policy (the probability of choosing an action), is crucial in causal inference. This paper proposes a DR estimator for dependent samples obtained from adaptive experiments. To obtain an asymptotically normal semiparametric estimator from dependent samples with non-Donsker nuisance estimators, we propose adaptive-fitting as a variant of sample-splitting. We also report an empirical paradox that our proposed DR estimator tends to show better performances compared to other estimators utilizing the true logging policy. While a similar phenomenon is known for estimators with i.i.d. samples, traditional explanations based on asymptotic efficiency cannot elucidate our case with dependent samples. We confirm this hypothesis through simulation studies.