A Survey of Automatic Hallucination Evaluation on Natural Language Generation
It addresses the critical problem of reliably assessing hallucinations in LLMs for ensuring model trustworthiness, but as a survey, it is incremental in synthesizing existing work rather than introducing new methods.
This survey tackles the fragmented methodologies in Automatic Hallucination Evaluation (AHE) for Large Language Models (LLMs) by analyzing 105 methods, revealing that 77.1% target LLMs and proposing a structured framework to organize the field and guide future research.
The rapid advancement of Large Language Models (LLMs) has brought a pressing challenge: how to reliably assess hallucinations to guarantee model trustworthiness. Although Automatic Hallucination Evaluation (AHE) has become an indispensable component of this effort, the field remains fragmented in its methodologies, limiting both conceptual clarity and practical progress. This survey addresses this critical gap through a systematic analysis of 105 evaluation methods, revealing that 77.1% specifically target LLMs, a paradigm shift that demands new evaluation frameworks. We formulate a structured framework to organize the field, based on a survey of foundational datasets and benchmarks and a taxonomy of evaluation methodologies, which together systematically document the evolution from pre-LLM to post-LLM approaches. Beyond taxonomical organization, we identify fundamental limitations in current approaches and their implications for real-world deployment. To guide future research, we delineate key challenges and propose strategic directions, including enhanced interpretability mechanisms and integration of application-specific evaluation criteria, ultimately providing a roadmap for developing more robust and practical hallucination evaluation systems.