Hierarchical Re-ranker Retriever (HRR)
This addresses the problem of context granularity in retrieval for LLM users, but it appears incremental as it builds on existing chunking and reranking methods.
The paper tackles the challenge of retrieving context at the right granularity for information retrieval by introducing the Hierarchical Re-ranker Retriever (HRR), which uses sentence-level and 512-token chunks with a reranker to balance specificity and context, mapping top results to 2048-token chunks for LLM applications.
Retrieving the right level of context for a given query is a perennial challenge in information retrieval - too large a chunk dilutes semantic specificity, while chunks that are too small lack broader context. This paper introduces the Hierarchical Re-ranker Retriever (HRR), a framework designed to achieve both fine-grained and high-level context retrieval for large language model (LLM) applications. In HRR, documents are split into sentence-level and intermediate-level (512 tokens) chunks to maximize vector-search quality for both short and broad queries. We then employ a reranker that operates on these 512-token chunks, ensuring an optimal balance neither too coarse nor too fine for robust relevance scoring. Finally, top-ranked intermediate chunks are mapped to parent chunks (2048 tokens) to provide an LLM with sufficiently large context.