A Representation Sharpening Framework for Zero Shot Dense Retrieval
This work addresses the challenge of improving retrieval accuracy in zero-shot settings for information retrieval systems, though it is incremental as it builds on prior approaches.
The paper tackles the problem of zero-shot dense retrieval where pretrained dense retrievers struggle to differentiate similar documents in an unseen corpus, and introduces a training-free representation sharpening framework that consistently outperforms traditional methods, setting a new state-of-the-art on the BRIGHT benchmark across over twenty multilingual datasets.
Zero-shot dense retrieval is a challenging setting where a document corpus is provided without relevant queries, necessitating a reliance on pretrained dense retrievers (DRs). However, since these DRs are not trained on the target corpus, they struggle to represent semantic differences between similar documents. To address this failing, we introduce a training-free representation sharpening framework that augments a document's representation with information that helps differentiate it from similar documents in the corpus. On over twenty datasets spanning multiple languages, the representation sharpening framework proves consistently superior to traditional retrieval, setting a new state-of-the-art on the BRIGHT benchmark. We show that representation sharpening is compatible with prior approaches to zero-shot dense retrieval and consistently improves their performance. Finally, we address the performance-cost tradeoff presented by our framework and devise an indexing-time approximation that preserves the majority of our performance gains over traditional retrieval, yet suffers no additional inference-time cost.