RepBERT: Contextualized Text Embeddings for First-Stage Retrieval
This addresses the limitation of exact term matching in retrieval systems, offering a novel approach for information retrieval tasks.
The paper tackled the problem of first-stage retrieval by proposing RepBERT, which uses contextualized embeddings for queries and documents, achieving state-of-the-art results on the MS MARCO Passage Ranking task with efficiency comparable to bag-of-words methods.
Although exact term match between queries and documents is the dominant method to perform first-stage retrieval, we propose a different approach, called RepBERT, to represent documents and queries with fixed-length contextualized embeddings. The inner products of query and document embeddings are regarded as relevance scores. On MS MARCO Passage Ranking task, RepBERT achieves state-of-the-art results among all initial retrieval techniques. And its efficiency is comparable to bag-of-words methods.