Data-Driven Off-Policy Estimator Selection: An Application in User Marketing on An Online Content Delivery Service
This work addresses a practical problem for practitioners in domains like marketing and healthcare, where accurate OPE is crucial to avoid deploying poor policies, but it is incremental as it builds on existing OPE methods by adding a selection framework.
The paper tackles the challenge of selecting the most suitable off-policy evaluation (OPE) estimator for specific applications, such as user marketing, by proposing a data-driven selection procedure and demonstrating its effectiveness in a real-world online content delivery service.
Off-policy evaluation (OPE) is the method that attempts to estimate the performance of decision making policies using historical data generated by different policies without conducting costly online A/B tests. Accurate OPE is essential in domains such as healthcare, marketing or recommender systems to avoid deploying poor performing policies, as such policies may hart human lives or destroy the user experience. Thus, many OPE methods with theoretical backgrounds have been proposed. One emerging challenge with this trend is that a suitable estimator can be different for each application setting. It is often unknown for practitioners which estimator to use for their specific applications and purposes. To find out a suitable estimator among many candidates, we use a data-driven estimator selection procedure for off-policy policy performance estimators as a practical solution. As proof of concept, we use our procedure to select the best estimator to evaluate coupon treatment policies on a real-world online content delivery service. In the experiment, we first observe that a suitable estimator might change with different definitions of the outcome variable, and thus the accurate estimator selection is critical in real-world applications of OPE. Then, we demonstrate that, by utilizing the estimator selection procedure, we can easily find out suitable estimators for each purpose.