FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering
This addresses the issue of unreliable LLM outputs for users in knowledge-intensive domains like question answering, though it is incremental as it builds on existing KG-enhanced methods.
The paper tackles the problem of LLMs generating erroneous or hallucinated responses in complex reasoning tasks by proposing FiDeLiS, a unified framework that anchors answers to verifiable reasoning steps retrieved from knowledge graphs, resulting in improved performance, factuality, and interpretability across benchmarks as a training-free method.
Large Language Models (LLMs) are often challenged by generating erroneous or hallucinated responses, especially in complex reasoning tasks. Leveraging Knowledge Graphs (KGs) as external knowledge sources has emerged as a viable solution. However, existing KG-enhanced methods, either retrieval-based or agent-based, encounter difficulties in accurately retrieving knowledge and efficiently traversing KGs at scale. In this paper, we propose a unified framework, FiDeLiS, designed to improve the factuality of LLM responses by anchoring answers to verifiable reasoning steps retrieved from KGs. To achieve this, we leverage step-wise beam search with a deductive scoring function, allowing the LLM to validate reasoning process step by step, and halt the search once the question is deducible. In addition, we propose a Path-RAG module to pre-select a smaller candidate set for each beam search step, reducing computational costs by narrowing the search space. Extensive experiments show that our method, as a training-free framework, not only improve the performance but also enhance the factuality and interpretability across different benchmarks. Code is released at https://github.com/Y-Sui/FiDeLiS.