Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions
For AI safety researchers, this work identifies and quantifies a fundamental failure mode in LLMs where reasoning fails to align values with actions, highlighting the need for new alignment techniques.
LLMs exhibit a value-action gap where stated values do not translate into actions, even under explicit reasoning, termed 'Pseudo-Deliberation.' The VALDI framework measures this misalignment across 4,941 scenarios, showing consistent gaps in both proprietary and open-source models.
Large language models (LLMs) are often evaluated based on their stated values, yet these do not reliably translate into their actions, a discrepancy termed "value-action gap." In this work, we argue that this gap persists even under explicit reasoning, revealing a deeper failure mode we call "Pseudo-Deliberation": the appearance of principled reasoning without corresponding behavioral alignment. To study this systematically, we introduce VALDI, a framework for measuring alignment between stated values and generated dialogue. VALDI includes 4,941 human-centered scenarios across five domains, three tasks that elicit value articulation, reasoning, and action, and five metrics for quantifying value adherence. Across both proprietary and open-source LLMs, we observe consistent misalignment between expressed values and downstream dialogues. To investigate intervention strategies, we propose VIVALDI, a multi-agent value auditor that intervenes at different stages of generation.