High Quality Related Search Query Suggestions using Deep Reinforcement Learning
This addresses the challenge of expensive human annotations and biases in query suggestion systems for search engine users, representing a strong domain-specific advancement.
The paper tackles the problem of generating high-quality related search query suggestions by training a Deep Reinforcement Learning model that predicts a user's next query using long-term session-based feedback, achieving relative improvements of 3% in diversity, 4.2% in user engagement, and 82% reduction in word repetitions compared to a baseline supervised model.
"High Quality Related Search Query Suggestions" task aims at recommending search queries which are real, accurate, diverse, relevant and engaging. Obtaining large amounts of query-quality human annotations is expensive. Prior work on supervised query suggestion models suffered from selection and exposure bias, and relied on sparse and noisy immediate user-feedback (e.g., clicks), leading to low quality suggestions. Reinforcement Learning techniques employed to reformulate a query using terms from search results, have limited scalability to large-scale industry applications. To recommend high quality related search queries, we train a Deep Reinforcement Learning model to predict the query a user would enter next. The reward signal is composed of long-term session-based user feedback, syntactic relatedness and estimated naturalness of generated query. Over the baseline supervised model, our proposed approach achieves a significant relative improvement in terms of recommendation diversity (3%), down-stream user-engagement (4.2%) and per-sentence word repetitions (82%).