MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models
This addresses the problem of improving LLM reliability and interpretability for users in domains like healthcare, though it is incremental as it builds on existing prompting and knowledge graph methods.
The authors tackled the limitations of large language models (LLMs) in incorporating new knowledge, generating hallucinations, and explaining reasoning by proposing a prompting pipeline that uses knowledge graphs to enhance inference and transparency, showing significant improvements on question-answering tasks, especially in medical domains.
Large language models (LLMs) have achieved remarkable performance in natural language understanding and generation tasks. However, they often suffer from limitations such as difficulty in incorporating new knowledge, generating hallucinations, and explaining their reasoning process. To address these challenges, we propose a novel prompting pipeline, named \method, that leverages knowledge graphs (KGs) to enhance LLMs' inference and transparency. Our method enables LLMs to comprehend KG inputs and infer with a combination of implicit and external knowledge. Moreover, our method elicits the mind map of LLMs, which reveals their reasoning pathways based on the ontology of knowledge. We evaluate our method on diverse question \& answering tasks, especially in medical domains, and show significant improvements over baselines. We also introduce a new hallucination evaluation benchmark and analyze the effects of different components of our method. Our results demonstrate the effectiveness and robustness of our method in merging knowledge from LLMs and KGs for combined inference. To reproduce our results and extend the framework further, we make our codebase available at https://github.com/wyl-willing/MindMap.