An Audit on the Perspectives and Challenges of Hallucinations in NLP
This work addresses the need for standardized definitions of hallucination in NLP, which is crucial for researchers and practitioners to mitigate societal impacts, though it is incremental as it synthesizes existing literature.
The authors audited 103 publications and surveyed 171 practitioners to analyze how hallucination is characterized in NLP literature, finding a lack of agreement on the term and calling for explicit definitions and frameworks to address societal challenges.
We audit how hallucination in large language models (LLMs) is characterized in peer-reviewed literature, using a critical examination of 103 publications across NLP research. Through the examination of the literature, we identify a lack of agreement with the term `hallucination' in the field of NLP. Additionally, to compliment our audit, we conduct a survey with 171 practitioners from the field of NLP and AI to capture varying perspectives on hallucination. Our analysis calls for the necessity of explicit definitions and frameworks outlining hallucination within NLP, highlighting potential challenges, and our survey inputs provide a thematic understanding of the influence and ramifications of hallucination in society.