Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Database
This addresses the need for more interpretable and comprehensive reasoning in textual question answering systems, though it is incremental as it builds on existing retrieval and reasoning models.
The paper tackles the problem of complex question answering requiring multi-chain and multi-hop retrieval by introducing Graph-Hop, a novel paradigm, and a new benchmark called ReasonGraphQA, which shows advantages in reasoning diversity and scale, along with a baseline method that was evaluated to highlight its promise despite limitations.
In Textual question answering (TQA) systems, complex questions often require retrieving multiple textual fact chains with multiple reasoning steps. While existing benchmarks are limited to single-chain or single-hop retrieval scenarios. In this paper, we propose to conduct Graph-Hop -- a novel multi-chains and multi-hops retrieval and reasoning paradigm in complex question answering. We construct a new benchmark called ReasonGraphQA, which provides explicit and fine-grained evidence graphs for complex questions to support interpretable reasoning, comprehensive and detailed reasoning. And ReasonGraphQA also shows an advantage in reasoning diversity and scale. Moreover, We propose a strong graph-hop baseline called Bidirectional Graph Retrieval (BGR) method for generating an explanation graph of textual evidence in knowledge reasoning and question answering. We have thoroughly evaluated existing evidence retrieval and reasoning models on the ReasonGraphQA. Experiments highlight Graph-Hop is a promising direction for answering complex questions, but it still has certain limitations. We have further studied mitigation strategies to meet these challenges and discuss future directions.