All Roads Lead to Rome: Graph-Based Confidence Estimation for Large Language Model Reasoning
This addresses the need for reliable LLM deployment in reasoning applications, though it is incremental as it adapts existing graph techniques to a new domain.
The paper tackled the problem of confidence estimation for large language models in reasoning tasks, proposing graph-based methods that improved estimation and downstream performance on three datasets.
Confidence estimation is essential for the reliable deployment of large language models (LLMs). Existing methods are primarily designed for factual QA tasks and often fail to generalize to reasoning tasks. To address this gap, we propose a set of training-free, graph-based confidence estimation methods tailored to reasoning tasks. Our approach models reasoning paths as directed graphs and estimates confidence by exploiting graph properties such as centrality, path convergence, and path weighting. Experiments with two LLMs on three reasoning datasets demonstrate improved confidence estimation and enhanced performance on two downstream tasks.