Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topology
This work addresses the reliability of LLM outputs for users needing interpretable AI, but it is incremental as it builds on existing uncertainty and explanation methods.
The authors tackled the problem of quantifying uncertainty in large language model (LLM) explanations by proposing a framework based on reasoning topology, which decomposes explanations into sub-questions and reasoning structures to assess uncertainty at semantic and path levels.
Understanding the uncertainty in large language model (LLM) explanations is important for evaluating their faithfulness and reasoning consistency, and thus provides insights into the reliability of LLM's output regarding a question. In this work, we propose a novel framework that quantifies uncertainty in LLM explanations through a reasoning topology perspective. By designing a structural elicitation strategy, we guide the LLMs to frame the explanations of an answer into a graph topology. This process decomposes the explanations into the knowledge related sub-questions and topology-based reasoning structures, which allows us to quantify uncertainty not only at the semantic level but also from the reasoning path. It further brings convenience to assess knowledge redundancy and provide interpretable insights into the reasoning process. Our method offers a systematic way to interpret the LLM reasoning, analyze limitations, and provide guidance for enhancing robustness and faithfulness. This work pioneers the use of graph-structured uncertainty measurement in LLM explanations and demonstrates the potential of topology-based quantification.