How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study
For researchers and developers of AI systems, this work provides neurocognitive insights into why humans are misled by AI hallucinations, highlighting the need for better detection and explanation mechanisms.
This study investigates how humans process AI-generated hallucinations by recording EEG signals from 27 participants during a verification task. It finds distinct neural patterns for hallucinated vs. non-hallucinated content, with misjudged hallucinations failing to trigger standard fact verification pathways.
While AI-generated hallucinations pose considerable risks, the underlying cognitive mechanisms by which humans can successfully recognize or be misled by these hallucinations remain unclear. To address this problem, this paper explores humans' neural dynamics to characterize how the brain processes hallucinated content. We record EEG signals from 27 participants while they are performing a verification task to judge the correctness of image descriptions generated by a multi-modal large language model (MLLM). Based on an averaged event-related potential (ERP) study, we reveal that multiple cognitive processes, e.g., semantic integration, inferential processing, memory retrieval, and cognitive load, exhibit distinct patterns when humans process hallucinated versus non-hallucinated content. Notably, neural responses to hallucinations that were misjudged versus correctly judged by human participants showed significant differences. This indicates that misjudged AI-generated hallucinations failed to trigger the standard neurocognitive fact verification pathway.