IntOPE: Off-Policy Evaluation in the Presence of Interference
This addresses a limitation in OPE methods for applications like personalized medicine and recommender systems where interference is common, representing an incremental improvement by extending existing IPW frameworks.
The paper tackles the problem of off-policy evaluation in contextual bandits when interference occurs, where an individual's reward is affected by others' actions, and proposes IntIPW, an estimator that accounts for this by integrating marginalized importance weights, showing effectiveness in experiments on synthetic and real-world data.
Off-Policy Evaluation (OPE) is employed to assess the potential impact of a hypothetical policy using logged contextual bandit feedback, which is crucial in areas such as personalized medicine and recommender systems, where online interactions are associated with significant risks and costs. Traditionally, OPE methods rely on the Stable Unit Treatment Value Assumption (SUTVA), which assumes that the reward for any given individual is unaffected by the actions of others. However, this assumption often fails in real-world scenarios due to the presence of interference, where an individual's reward is affected not just by their own actions but also by the actions of their peers. This realization reveals significant limitations of existing OPE methods in real-world applications. To address this limitation, we propose IntIPW, an IPW-style estimator that extends the Inverse Probability Weighting (IPW) framework by integrating marginalized importance weights to account for both individual actions and the influence of adjacent entities. Extensive experiments are conducted on both synthetic and real-world data to demonstrate the effectiveness of the proposed IntIPW method.