KDCM: Reducing Hallucination in LLMs through Explicit Reasoning Structures
This addresses the issue of unreliable predictions in LLMs for users needing accurate and interpretable outputs, though it appears incremental as it builds on existing chain-style knowledge distillation methods.
The paper tackles the problem of hallucinations in large language models (LLMs) by proposing a framework that uses explicit reasoning structures with code-guided knowledge graph exploration, resulting in significant improvements in metrics like HIT@1 increasing by 15.64% and scores exceeding 95% in evaluations.
To mitigate hallucinations in large language models (LLMs), we propose a framework that focuses on errors induced by prompts. Our method extends a chain-style knowledge distillation approach by incorporating a programmable module that guides knowledge graph exploration. This module is embedded as executable code within the reasoning prompt, allowing the model to leverage external structured knowledge during inference. Based on this design, we develop an enhanced distillation-based reasoning framework that explicitly regulates intermediate reasoning steps, resulting in more reliable predictions. We evaluate the proposed approach on multiple public benchmarks using GPT-4 and LLaMA-3.3. Experimental results show that code-guided reasoning significantly improves contextual modeling and reduces prompt-induced hallucinations. Specifically, HIT@1, HIT@3, and HIT@5 increase by 15.64%, 13.38%, and 13.28%, respectively, with scores exceeding 95% across several evaluation settings. These findings indicate that the proposed method effectively constrains erroneous reasoning while improving both accuracy and interpretability.