Three Sides of Retrieval: Factorial Evidence for Document-Side, Query-Side, and Answer-Side Complementarity in RAG
For practitioners building RAG systems on long documents, this work offers a cheap retrieval improvement and demonstrates that all three sides of retrieval (document, query, answer) can be combined for better performance.
The paper introduces a zero-LLM-cost ToC-guided page retrieval method for RAG that infers headings from visual formatting, and provides factorial evidence that document-side, query-side, and answer-side enhancements are complementary, with ToC retrieval improving answer quality (d=+0.41) and contributing 20% of citations while adding only 2.9 pages per query.
RAG systems rely on chunking, which destroys structural information in documents. Existing heading-based retrieval (Jeong et al., 2025) requires multiple LLM calls per document and returns sub-chunks within matched sections. We introduce ToC-guided page retrieval, which infers headings from visual formatting without LLM calls, embeds them as a parallel index, and loads full page sections. Across 1,280 conditions on 8 enterprise documents (5 to 195 pages), we find: (1) ToC is a significant main effect on answer quality (d = +0.41, p = 0.031), with the largest gains in completeness (+0.40) and usefulness (+0.40); (2) combined with answer-side verification, it outperforms query-side decomposition + verification (d = +0.32, p = 0.036); (3) ToC contributes 20% of citations despite adding only 2.9 pages per query; (4) gains are directionally larger on longer documents (up to +1.50 on 118 pages), though the trend does not reach significance with 8 documents; and (5) a 480-condition sensitivity analysis finds no significant parameter effects (all p > 0.38), confirming defaults are near-optimal. The contribution is both methodological (a new zero-LLM-cost retrieval algorithm) and empirical: factorial evidence that document-side, query-side, and answer-side enhancements are complementary, a three-way interaction not previously studied.