First Hallucination Tokens Are Different from Conditional Ones
This work addresses the problem of fine-grained hallucination detection for improving trust in LLMs, though it is incremental as it builds on existing token-level detection approaches.
The paper investigates token-level hallucination detection in Large Language Models, finding that the first hallucinated token is significantly more detectable than subsequent ones, with this structural property holding consistently across different models.
Large Language Models (LLMs) hallucinate, and detecting these cases is key to ensuring trust. While many approaches address hallucination detection at the response or span level, recent work explores token-level detection, enabling more fine-grained intervention. However, the distribution of hallucination signal across sequences of hallucinated tokens remains unexplored. We leverage token-level annotations from the RAGTruth corpus and find that the first hallucinated token is far more detectable than later ones. This structural property holds across models, suggesting that first hallucination tokens play a key role in token-level hallucination detection. Our code is available at https://github.com/jakobsnl/RAGTruth_Xtended.