Neural Document Expansion with User Feedback
This work addresses the challenge of enhancing document retrieval for search engines, though it is incremental as it builds on existing neural ranking and expansion techniques.
The paper tackles the problem of improving neural ranking models by proposing NeuDEF, a neural document expansion method that uses click-based query terms and learned attention weights, which significantly boosts the accuracy of state-of-the-art rankers across various query frequencies in a commercial search log.
This paper presents a neural document expansion approach (NeuDEF) that enriches document representations for neural ranking models. NeuDEF harvests expansion terms from queries which lead to clicks on the document and weights these expansion terms with learned attention. It is plugged into a standard neural ranker and learned end-to-end. Experiments on a commercial search log demonstrate that NeuDEF significantly improves the accuracy of state-of-the-art neural rankers and expansion methods on queries with different frequencies. Further studies show the contribution of click queries and learned expansion weights, as well as the influence of document popularity of NeuDEF's effectiveness.