Efficient and Effective Query Context-Aware Learning-to-Rank Model for Sequential Recommendation
This work addresses the problem of enhancing user engagement in sequential recommendation systems by refining ranking with query context, representing an incremental improvement over existing transformer methods.
The paper tackled the challenge of integrating query context into transformer-based sequential recommenders to improve ranking relevance and diversity, proposing a method that fuses item sequences with query context in the attention mechanism and demonstrating effectiveness through offline and online experiments on large-scale datasets.
Modern sequential recommender systems commonly use transformer-based models for next-item prediction. While these models demonstrate a strong balance between efficiency and quality, integrating interleaving features - such as the query context (e.g., browse category) under which next-item interactions occur - poses challenges. Effectively capturing query context is crucial for refining ranking relevance and enhancing user engagement, as it provides valuable signals about user intent within a session. Unlike item features, historical query context is typically not aligned with item sequences and may be unavailable at inference due to privacy constraints or feature store limitations - making its integration into transformers both challenging and error-prone. This paper analyzes different strategies for incorporating query context into transformers trained with a causal language modeling procedure as a case study. We propose a new method that effectively fuses the item sequence with query context within the attention mechanism. Through extensive offline and online experiments on a large-scale online platform and open datasets, we present evidence that our proposed method is an effective approach for integrating query context to improve model ranking quality in terms of relevance and diversity.