Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering
This addresses a critical bottleneck in multi-hop QA systems for users relying on accurate information retrieval, though it appears incremental as it builds on existing retrieval-augmented methods.
The paper tackled the 'lost-in-retrieval' problem in retrieval-augmented multi-hop question answering, where missing key entities degrade retrieval and reasoning, and proposed ChainRAG, a progressive retrieval and rewriting method that improved performance on datasets like MuSiQue, 2Wiki, and HotpotQA, outperforming baselines in effectiveness and efficiency.
In this paper, we identify a critical problem, "lost-in-retrieval", in retrieval-augmented multi-hop question answering (QA): the key entities are missed in LLMs' sub-question decomposition. "Lost-in-retrieval" significantly degrades the retrieval performance, which disrupts the reasoning chain and leads to the incorrect answers. To resolve this problem, we propose a progressive retrieval and rewriting method, namely ChainRAG, which sequentially handles each sub-question by completing missing key entities and retrieving relevant sentences from a sentence graph for answer generation. Each step in our retrieval and rewriting process builds upon the previous one, creating a seamless chain that leads to accurate retrieval and answers. Finally, all retrieved sentences and sub-question answers are integrated to generate a comprehensive answer to the original question. We evaluate ChainRAG on three multi-hop QA datasets - MuSiQue, 2Wiki, and HotpotQA - using three large language models: GPT4o-mini, Qwen2.5-72B, and GLM-4-Plus. Empirical results demonstrate that ChainRAG consistently outperforms baselines in both effectiveness and efficiency.