A Survey of Uncertainty Estimation Methods on Large Language Models
It addresses the problem of unreliable LLM outputs for researchers and practitioners, but it is incremental as a survey rather than a novel method.
This survey tackles the lack of comprehensive reviews on uncertainty estimation methods for large language models (LLMs), which are prone to biased or hallucinated responses, by categorizing four major approaches and evaluating them across multiple datasets.
Large language models (LLMs) have demonstrated remarkable capabilities across various tasks. However, these models could offer biased, hallucinated, or non-factual responses camouflaged by their fluency and realistic appearance. Uncertainty estimation is the key method to address this challenge. While research efforts in uncertainty estimation are ramping up, there is a lack of comprehensive and dedicated surveys on LLM uncertainty estimation. This survey presents four major avenues of LLM uncertainty estimation. Furthermore, we perform extensive experimental evaluations across multiple methods and datasets. At last, we provide critical and promising future directions for LLM uncertainty estimation.