Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective
This work addresses the need for trustworthy AI in high-stakes applications by improving uncertainty estimation in KG-LLM systems, though it is incremental as it builds on existing frameworks.
The paper tackles the problem of unreliable knowledge graph reasoning with large language models by proposing an uncertainty-aware framework that incorporates conformal prediction, achieving a 40% reduction in prediction set size while maintaining any pre-defined coverage rate.
Recently, Knowledge Graphs (KGs) have been successfully coupled with Large Language Models (LLMs) to mitigate their hallucinations and enhance their reasoning capability, such as in KG-based retrieval-augmented frameworks. However, current KG-LLM frameworks lack rigorous uncertainty estimation, limiting their reliable deployment in high-stakes applications. Directly incorporating uncertainty quantification into KG-LLM frameworks presents challenges due to their complex architectures and the intricate interactions between the knowledge graph and language model components. To address this gap, we propose a new trustworthy KG-LLM framework, Uncertainty Aware Knowledge-Graph Reasoning (UAG), which incorporates uncertainty quantification into the KG-LLM framework. We design an uncertainty-aware multi-step reasoning framework that leverages conformal prediction to provide a theoretical guarantee on the prediction set. To manage the error rate of the multi-step process, we additionally introduce an error rate control module to adjust the error rate within the individual components. Extensive experiments show that our proposed UAG can achieve any pre-defined coverage rate while reducing the prediction set/interval size by 40% on average over the baselines.