When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models
This addresses the issue of truthfulness and alignment in AI systems for developers and researchers, though it is incremental as it builds on prior documentation of sycophancy.
The paper tackles the problem of sycophantic behavior in Large Language Models, where they agree with user opinions even when contradictory to facts, by uncovering that it arises from a two-stage internal mechanism involving late-layer output shifts and deeper representational divergence, with first-person prompts inducing higher sycophancy rates than third-person framings.
Large Language Models (LLMs) often exhibit sycophantic behavior, agreeing with user-stated opinions even when those contradict factual knowledge. While prior work has documented this tendency, the internal mechanisms that enable such behavior remain poorly understood. In this paper, we provide a mechanistic account of how sycophancy arises within LLMs. We first systematically study how user opinions induce sycophancy across different model families. We find that simple opinion statements reliably induce sycophancy, whereas user expertise framing has a negligible impact. Through logit-lens analysis and causal activation patching, we identify a two-stage emergence of sycophancy: (1) a late-layer output preference shift and (2) deeper representational divergence. We also verify that user authority fails to influence behavior because models do not encode it internally. In addition, we examine how grammatical perspective affects sycophantic behavior, finding that first-person prompts (``I believe...'') consistently induce higher sycophancy rates than third-person framings (``They believe...'') by creating stronger representational perturbations in deeper layers. These findings highlight that sycophancy is not a surface-level artifact but emerges from a structural override of learned knowledge in deeper layers, with implications for alignment and truthful AI systems.