Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior
For AI safety researchers, it reveals a new vulnerability where reasoning traces can be reused as jailbreak attacks, motivating defenses that evaluate reasoning context.
The paper shows that harmful chain-of-thought reasoning traces from compromised language models can transfer unsafe behavior to other models, achieving over 80% harmful-response rates on vulnerable open-source models, and that distilled patterns into system prompts outperform direct transplantation by up to 10× on GPT-4.1 AdvBench.
We investigate whether harmful chain-of-thought (CoT) traces from compromised language models can transfer unsafe behaviour and be distilled into reusable jailbreak attacks. Using an emergent-misalignment organism and a refusal-ablated jailbroken organism, we transplant harmful CoTs into $29$ open-source and $5$ closed-source targets. Transferred traces raise harmful-response rates above $80\%$ on the most vulnerable open-source models, while semantically mismatched CoTs fail entirely. LLooM concept mining identifies four recurring components of harmful reasoning: proceduralisation, ethical decoupling, evasion, and target--vulnerability framing. Distilling these patterns into reusable system prompts produces effective black-box jailbreaks, outperforming direct CoT transplantation on strongly aligned models by up to an order of magnitude, including a $10\times$ improvement on GPT-4.1 AdvBench. Reasoning-enabled models are more than twice as vulnerable, and output-side safeguards such as Llama-Guard~3 frequently miss harmful generations. Our results show that harmful reasoning transfers at both the trace and pattern levels, motivating defences that evaluate reasoning context in addition to final outputs.