U-rank: Utility-oriented Learning to Rank with Implicit Feedback
This work addresses the utility optimization problem in ranking for information systems like web search and recommender systems, offering a novel approach that improves over existing methods.
The paper tackles the problem of learning to rank with implicit feedback by proposing U-rank, a framework that directly optimizes expected utility, achieving significant performance gains over state-of-the-art methods in experiments on multiple datasets and showing large improvements in online A/B testing on a commercial recommender system.
Learning to rank with implicit feedback is one of the most important tasks in many real-world information systems where the objective is some specific utility, e.g., clicks and revenue. However, we point out that existing methods based on probabilistic ranking principle do not necessarily achieve the highest utility. To this end, we propose a novel ranking framework called U-rank that directly optimizes the expected utility of the ranking list. With a position-aware deep click-through rate prediction model, we address the attention bias considering both query-level and item-level features. Due to the item-specific attention bias modeling, the optimization for expected utility corresponds to a maximum weight matching on the item-position bipartite graph. We base the optimization of this objective in an efficient Lambdaloss framework, which is supported by both theoretical and empirical analysis. We conduct extensive experiments for both web search and recommender systems over three benchmark datasets and two proprietary datasets, where the performance gain of U-rank over state-of-the-arts is demonstrated. Moreover, our proposed U-rank has been deployed on a large-scale commercial recommender and a large improvement over the production baseline has been observed in an online A/B testing.