Graph-based Confidence Calibration for Large Language Models
This work addresses the challenge of enhancing trustworthiness in LLMs for high-stakes scenarios, representing an incremental improvement in confidence calibration methods.
The paper tackles the problem of reliable confidence estimation for large language models (LLMs) by proposing a method that uses a consistency graph and graph neural network to assess response correctness based on self-consistency of multiple outputs, achieving strong calibration performance on benchmark datasets and good generalization to out-of-domain cases.
Reliable confidence estimation is essential for enhancing the trustworthiness of large language models (LLMs), especially in high-stakes scenarios. Despite its importance, accurately estimating confidence in LLM responses remains a significant challenge. In this work, we propose using an auxiliary learning model to assess response correctness based on the self-consistency of multiple outputs generated by the LLM. Our method builds a consistency graph to represent the agreement among multiple responses and uses a graph neural network (GNN) to estimate the likelihood that each response is correct. Experiments demonstrate that this method has strong calibration performance on various benchmark datasets and generalizes well to out-of-domain cases.