The benefits of query-based KGQA systems for complex and temporal questions in LLM era
This work addresses complex reasoning tasks in QA for applications requiring accurate temporal and multi-hop answers, but it is incremental as it builds on existing query-based KGQA methods.
The paper tackles the problem of multi-hop and temporal question-answering where large language models struggle, by proposing a multi-stage query-based knowledge graph QA framework that improves performance on challenging benchmarks, demonstrating its potential with small language models.
Large language models excel in question-answering (QA) yet still struggle with multi-hop reasoning and temporal questions. Query-based knowledge graph QA (KGQA) offers a modular alternative by generating executable queries instead of direct answers. We explore multi-stage query-based framework for WikiData QA, proposing multi-stage approach that enhances performance on challenging multi-hop and temporal benchmarks. Through generalization and rejection studies, we evaluate robustness across multi-hop and temporal QA datasets. Additionally, we introduce a novel entity linking and predicate matching method using CoT reasoning. Our results demonstrate the potential of query-based multi-stage KGQA framework for improving multi-hop and temporal QA with small language models. Code and data: https://github.com/ar2max/NLDB-KGQA-System