Disentangling Deception and Hallucination Failures in LLMs
This work addresses the need for more precise failure analysis in LLMs for researchers and developers, though it is incremental as it builds on existing behavioral perspectives.
The paper tackles the problem of conflated failure mechanisms in LLMs by proposing a distinction between hallucination and deception based on internal mechanisms, and constructs a controlled environment to analyze these failures through representation separability and activation steering.
Failures in large language models (LLMs) are often analyzed from a behavioral perspective, where incorrect outputs in factual question answering are commonly associated with missing knowledge. In this work, focusing on entity-based factual queries, we suggest that such a view may conflate different failure mechanisms, and propose an internal, mechanism-oriented perspective that separates Knowledge Existence from Behavior Expression. Under this formulation, hallucination and deception correspond to two qualitatively different failure modes that may appear similar at the output level but differ in their underlying mechanisms. To study this distinction, we construct a controlled environment for entity-centric factual questions in which knowledge is preserved while behavioral expression is selectively altered, enabling systematic analysis of four behavioral cases. We analyze these failure modes through representation separability, sparse interpretability, and inference-time activation steering.