Counterfactual Evaluation of Ads Ranking Models through Domain Adaptation
This work provides a more effective evaluation method for ads ranking models in large-scale recommender systems, which is crucial for advertisers and platform providers.
This paper addresses the challenge of evaluating ads ranking models in large-scale recommender systems by proposing a domain-adapted reward model. This model, integrated with an Offline A/B testing system, effectively measures reward for ranking model changes and outperforms both the vanilla IPS method and non-generalized reward models.
We propose a domain-adapted reward model that works alongside an Offline A/B testing system for evaluating ranking models. This approach effectively measures reward for ranking model changes in large-scale Ads recommender systems, where model-free methods like IPS are not feasible. Our experiments demonstrate that the proposed technique outperforms both the vanilla IPS method and approaches using non-generalized reward models.