Distributional Off-Policy Evaluation for Slate Recommendations
This work addresses a critical bottleneck in evaluating recommendation systems for slates, enabling more comprehensive assessments of risk and fairness, though it is incremental as it builds upon prior methods in off-policy evaluation.
The paper tackles the problem of estimating the complete performance distribution for slate recommendation strategies in off-policy evaluation, where combinatorial action spaces make existing methods impractical. The proposed estimator is shown to be unbiased and consistent, with empirical results on synthetic and real-world data demonstrating significant reductions in estimation variance and improved sample efficiency compared to prior work.
Recommendation strategies are typically evaluated by using previously logged data, employing off-policy evaluation methods to estimate their expected performance. However, for strategies that present users with slates of multiple items, the resulting combinatorial action space renders many of these methods impractical. Prior work has developed estimators that leverage the structure in slates to estimate the expected off-policy performance, but the estimation of the entire performance distribution remains elusive. Estimating the complete distribution allows for a more comprehensive evaluation of recommendation strategies, particularly along the axes of risk and fairness that employ metrics computable from the distribution. In this paper, we propose an estimator for the complete off-policy performance distribution for slates and establish conditions under which the estimator is unbiased and consistent. This builds upon prior work on off-policy evaluation for slates and off-policy distribution estimation in reinforcement learning. We validate the efficacy of our method empirically on synthetic data as well as on a slate recommendation simulator constructed from real-world data (MovieLens-20M). Our results show a significant reduction in estimation variance and improved sample efficiency over prior work across a range of slate structures.