Off-Policy Evaluation for Ranking Policies under Deterministic Logging Policies
This addresses a critical bottleneck in algorithmic ranking systems by enabling accurate performance estimation under deterministic logging policies, which is an incremental improvement over existing stochastic methods.
The paper tackles the problem of off-policy evaluation for ranking policies when the logging policy is deterministic, which causes severe bias in existing estimators, and proposes Click-based Inverse Propensity Score (CIPS) estimators that achieve significantly lower bias in synthetic and real-world experiments.
Off-Policy Evaluation (OPE) is an important practical problem in algorithmic ranking systems, where the goal is to estimate the expected performance of a new ranking policy using only offline logged data collected under a different, logging policy. Existing estimators, such as the ranking-wise and position-wise inverse propensity score (IPS) estimators, require the data collection policy to be sufficiently stochastic and suffer from severe bias when the logging policy is fully deterministic. In this paper, we propose novel estimators, Click-based Inverse Propensity Score (CIPS), exploiting the intrinsic stochasticity of user click behavior to address this challenge. Unlike existing methods that rely on the stochasticity of the logging policy, our approach uses click probability as a new form of importance weighting, enabling low-bias OPE even under deterministic logging policies where existing methods incur substantial bias. We provide theoretical analyses of the bias and variance properties of the proposed estimators and show, through synthetic and real-world experiments, that our estimators achieve significantly lower bias compared to strong baselines, for a range of experimental settings with completely deterministic logging policies.