Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generation
This work addresses the need for reliable confidence measures in LLMs for applications like question-answering, though it is incremental as it builds on existing sequence probability methods.
The paper tackles the problem of inaccurate confidence scores in large language models (LLMs) for natural language generation by proposing Contextualized Sequence Likelihood (CSL), which enhances sequence probability with token weights from attention heads, resulting in significantly higher reliability than state-of-the-art baselines across multiple QA datasets and LLMs as measured by AUROC or AUARC.
The advent of large language models (LLMs) has dramatically advanced the state-of-the-art in numerous natural language generation tasks. For LLMs to be applied reliably, it is essential to have an accurate measure of their confidence. Currently, the most commonly used confidence score function is the likelihood of the generated sequence, which, however, conflates semantic and syntactic components. For instance, in question-answering (QA) tasks, an awkward phrasing of the correct answer might result in a lower probability prediction. Additionally, different tokens should be weighted differently depending on the context. In this work, we propose enhancing the predicted sequence probability by assigning different weights to various tokens using attention values elicited from the base LLM. By employing a validation set, we can identify the relevant attention heads, thereby significantly improving the reliability of the vanilla sequence probability confidence measure. We refer to this new score as the Contextualized Sequence Likelihood (CSL). CSL is easy to implement, fast to compute, and offers considerable potential for further improvement with task-specific prompts. Across several QA datasets and a diverse array of LLMs, CSL has demonstrated significantly higher reliability than state-of-the-art baselines in predicting generation quality, as measured by the AUROC or AUARC.