Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry
For AI safety researchers, this provides a robust method to mitigate intrinsic deception in LLMs that is resistant to semantic concealment, addressing a critical trustworthiness bottleneck.
The paper identifies that deceptive LLMs exhibit stable internal chain-of-thought but fragile external responses under perturbation, a phenomenon called stability asymmetry. The proposed Stability Asymmetry Regularization (SAR) reduces deception by penalizing this asymmetry, achieving effective deception suppression without harming general performance.
As Large Language Models (LLMs) expand in capability and application scope, their trustworthiness becomes critical. A vital risk is intrinsic deception, wherein models strategically mislead users to achieve their own objectives. Existing alignment approaches based on chain-of-thought (CoT) monitoring supervise explicit reasoning traces. However, under optimization pressure, models are incentivized to conceal deceptive reasoning, rendering semantic supervision fundamentally unreliable. Grounded in cognitive psychology, we hypothesize that a deceptive LLM maintains a stable internal belief in its CoT while its external response remains fragile under perturbation. We term this phenomenon stability asymmetry and quantify it by measuring the contrast between internal CoT stability and external response stability under perturbation. Building on this structural signature, we propose the Stability Asymmetry Regularization (SAR), a novel alignment objective that penalizes this distributional asymmetry during reinforcement learning. Unlike CoT monitoring, SAR targets the statistical structure of model outputs, rendering it robust to semantic concealment. Extensive experiments confirm that stability asymmetry reliably identifies deceptive behavior, and that SAR effectively suppresses intrinsic deception without degrading general model capability.