Learning Passage Impacts for Inverted Indexes
This work addresses efficiency bottlenecks in retrieval systems for users needing faster and more accurate document search, though it is incremental as it builds on existing methods like DocT5Query.
The paper tackles the problem of improving first-stage retrieval efficiency in neural information retrieval by proposing DeepImpact, a document term-weighting scheme that enhances impact-score modeling and addresses vocabulary mismatch, resulting in up to 17% better effectiveness and 5.1x speedup in re-ranking scenarios.
Neural information retrieval systems typically use a cascading pipeline, in which a first-stage model retrieves a candidate set of documents and one or more subsequent stages re-rank this set using contextualized language models such as BERT. In this paper, we propose DeepImpact, a new document term-weighting scheme suitable for efficient retrieval using a standard inverted index. Compared to existing methods, DeepImpact improves impact-score modeling and tackles the vocabulary-mismatch problem. In particular, DeepImpact leverages DocT5Query to enrich the document collection and, using a contextualized language model, directly estimates the semantic importance of tokens in a document, producing a single-value representation for each token in each document. Our experiments show that DeepImpact significantly outperforms prior first-stage retrieval approaches by up to 17% on effectiveness metrics w.r.t. DocT5Query, and, when deployed in a re-ranking scenario, can reach the same effectiveness of state-of-the-art approaches with up to 5.1x speedup in efficiency.