Beyond the Reranker: Do RAG Retrieval Enhancements Help Once a Strong Reranker Is Present?
For practitioners building RAG systems, the paper shows that once a strong reranker is present, many popular retrieval enhancements are unnecessary, saving computational cost and complexity.
The paper investigates whether common RAG retrieval enhancements (e.g., query expansion, hierarchical summarization, graph expansion) improve performance when a strong cross-encoder reranker is already used. On a new heterogeneous benchmark (HetDocQA) and homogeneous controls, only query expansion and the proposed SSCC method yield reliable gains; most other methods provide no significant benefit.
Retrieval-augmented generation (RAG) is routinely extended with methods meant to improve retrieval: query expansion, hierarchical and cross-document summarization, graph-based expansion, per-query routing, rank fusion, and corrective re-retrieval. The benefits reported for these methods come almost exclusively from homogeneous corpora, predominantly Wikipedia prose. Whether they hold on the mixed-format collections common in practice, where code, markdown, tables, scientific PDFs, and prose are interleaved within one corpus, has not been measured. To study this directly, we build \textbf{HetDocQA}, a heterogeneous benchmark with \emph{chunker-agnostic} span-overlap relevance labels and collection-disjoint splits, and pair it with MuSiQue and QASPER as homogeneous controls. We evaluate eight methods on a shared backbone, with bootstrap confidence intervals and multiple-comparison correction. A strong cross-encoder reranker accounts for most of the pipeline's quality; beyond it, only two methods yield reliable gains: query expansion and SSCC. SSCC, a per-source calibrated corrector introduced here, sets a separate acceptance threshold for each score source and helps only on heterogeneous data. The remaining reranking and pool-expansion methods in common use, among them hierarchical summarization, graph expansion, routing, and rank fusion, give no reliable gain once that reranker is present.