Conflicts Make Large Reasoning Models Vulnerable to Attacks
For developers and users of LRMs, this work reveals a critical vulnerability in safety alignment under conflicts, highlighting the need for more robust alignment strategies.
Large Reasoning Models (LRMs) are vulnerable to attacks when faced with conflicting objectives, with conflict-based prompts increasing attack success rates by up to 30% across three models (Llama-3.1-Nemotron-8B, QwQ-32B, DeepSeek R1) on over 1,300 prompts.
Large Reasoning Models (LRMs) have achieved remarkable performance across diverse domains, yet their decision-making under conflicting objectives remains insufficiently understood. This work investigates how LRMs respond to harmful queries when confronted with two categories of conflicts: internal conflicts that pit alignment values against each other and dilemmas, which impose mutually contradictory choices, including sacrificial, duress, agent-centered, and social forms. Using over 1,300 prompts across five benchmarks, we evaluate three representative LRMs - Llama-3.1-Nemotron-8B, QwQ-32B, and DeepSeek R1 - and find that conflicts significantly increase attack success rates, even under single-round non-narrative queries without sophisticated auto-attack techniques. Our findings reveal through layerwise and neuron-level analyses that safety-related and functional representations shift and overlap under conflict, interfering with safety-aligned behavior. This study highlights the need for deeper alignment strategies to ensure the robustness and trustworthiness of next-generation reasoning models. Our code is available at https://github.com/DataArcTech/ConflictHarm. Warning: This paper contains inappropriate, offensive and harmful content.