Hallucination is Inevitable for LLMs with the Open World Assumption
This work addresses the fundamental challenge of hallucination in LLMs for AGI development, suggesting a shift from elimination to management, but it is incremental as it builds on existing theoretical analyses.
The paper reframes LLM hallucination as a generalization problem, arguing that under the open-world assumption where environments are unbounded, hallucinations become inevitable rather than just a defect to be minimized.
Large Language Models (LLMs) exhibit impressive linguistic competence but also produce inaccurate or fabricated outputs, often called ``hallucinations''. Engineering approaches usually regard hallucination as a defect to be minimized, while formal analyses have argued for its theoretical inevitability. Yet both perspectives remain incomplete when considering the conditions required for artificial general intelligence (AGI). This paper reframes ``hallucination'' as a manifestation of the generalization problem. Under the Closed World assumption, where training and test distributions are consistent, hallucinations may be mitigated. Under the Open World assumption, however, where the environment is unbounded, hallucinations become inevitable. This paper further develops a classification of hallucination, distinguishing cases that may be corrected from those that appear unavoidable under open-world conditions. On this basis, it suggests that ``hallucination'' should be approached not merely as an engineering defect but as a structural feature to be tolerated and made compatible with human intelligence.