Debiased Off-Policy Evaluation for Recommendation Systems
This provides a more efficient and reliable alternative to A/B testing for improving recommendation and advertisement systems, though it is incremental as it builds on existing off-policy evaluation methods.
The paper tackles the problem of evaluating new algorithms in interactive systems like recommendation systems without costly A/B tests by developing an off-policy estimator that predicts performance from historical data generated by different algorithms, achieving convergence at a rate of √N and producing smaller mean squared errors than state-of-the-art methods in simulations and real-world applications.
Efficient methods to evaluate new algorithms are critical for improving interactive bandit and reinforcement learning systems such as recommendation systems. A/B tests are reliable, but are time- and money-consuming, and entail a risk of failure. In this paper, we develop an alternative method, which predicts the performance of algorithms given historical data that may have been generated by a different algorithm. Our estimator has the property that its prediction converges in probability to the true performance of a counterfactual algorithm at a rate of $\sqrt{N}$, as the sample size $N$ increases. We also show a correct way to estimate the variance of our prediction, thus allowing the analyst to quantify the uncertainty in the prediction. These properties hold even when the analyst does not know which among a large number of potentially important state variables are actually important. We validate our method by a simulation experiment about reinforcement learning. We finally apply it to improve advertisement design by a major advertisement company. We find that our method produces smaller mean squared errors than state-of-the-art methods.