Zero-Shot Listwise Document Reranking with a Large Language Model
This addresses the need for efficient and effective document retrieval in web search without requiring large labeled datasets, though it is incremental as it builds on existing ranking methods.
The paper tackles the problem of document reranking without task-specific training data by proposing a listwise reranking method using a large language model, which outperforms zero-shot pointwise methods on TREC datasets and shows potential for multilingual generalization on MIRACL subsets.
Supervised ranking methods based on bi-encoder or cross-encoder architectures have shown success in multi-stage text ranking tasks, but they require large amounts of relevance judgments as training data. In this work, we propose Listwise Reranker with a Large Language Model (LRL), which achieves strong reranking effectiveness without using any task-specific training data. Different from the existing pointwise ranking methods, where documents are scored independently and ranked according to the scores, LRL directly generates a reordered list of document identifiers given the candidate documents. Experiments on three TREC web search datasets demonstrate that LRL not only outperforms zero-shot pointwise methods when reranking first-stage retrieval results, but can also act as a final-stage reranker to improve the top-ranked results of a pointwise method for improved efficiency. Additionally, we apply our approach to subsets of MIRACL, a recent multilingual retrieval dataset, with results showing its potential to generalize across different languages.