Reason-Align-Respond: Aligning LLM Reasoning with Knowledge Graphs for KGQA
This addresses the problem of unreliable factual grounding in LLMs for KGQA, offering a novel integration method that is incremental in combining existing techniques.
The paper tackles the problem of hallucinations and lack of factual grounding in LLMs for knowledge graph question answering (KGQA) by introducing the Reason-Align-Respond (RAR) framework, which integrates LLM reasoning with knowledge graphs and achieves state-of-the-art performance with Hit@1 scores of 93.3% and 91.0% on WebQSP and CWQ benchmarks.
LLMs have demonstrated remarkable capabilities in complex reasoning tasks, yet they often suffer from hallucinations and lack reliable factual grounding. Meanwhile, knowledge graphs (KGs) provide structured factual knowledge but lack the flexible reasoning abilities of LLMs. In this paper, we present Reason-Align-Respond (RAR), a novel framework that systematically integrates LLM reasoning with knowledge graphs for KGQA. Our approach consists of three key components: a Reasoner that generates human-like reasoning chains, an Aligner that maps these chains to valid KG paths, and a Responser that synthesizes the final answer. We formulate this process as a probabilistic model and optimize it using the Expectation-Maximization algorithm, which iteratively refines the reasoning chains and knowledge paths. Extensive experiments on multiple benchmarks demonstrate the effectiveness of RAR, achieving state-of-the-art performance with Hit@1 scores of 93.3% and 91.0% on WebQSP and CWQ respectively. Human evaluation confirms that RAR generates high-quality, interpretable reasoning chains well-aligned with KG paths. Furthermore, RAR exhibits strong zero-shot generalization capabilities and maintains computational efficiency during inference.