On the Role of Unobserved Sequences on Sample-based Uncertainty Quantification for LLMs
This work addresses uncertainty quantification for LLMs to improve reliability in safety-critical scenarios, but it is incremental as it builds on existing entropy-based methods.
The paper tackles the problem of uncertainty quantification in large language models (LLMs) for safety-critical applications by highlighting the importance of unobserved sequences in entropy estimation, and it experimentally demonstrates that integrating these sequences can enhance such methods.
Quantifying uncertainty in large language models (LLMs) is important for safety-critical applications because it helps spot incorrect answers, known as hallucinations. One major trend of uncertainty quantification methods is based on estimating the entropy of the distribution of the LLM's potential output sequences. This estimation is based on a set of output sequences and associated probabilities obtained by querying the LLM several times. In this paper, we advocate and experimentally show that the probability of unobserved sequences plays a crucial role, and we recommend future research to integrate it to enhance such LLM uncertainty quantification methods.