The Cost of Thinking: Increased Jailbreak Risk in Large Language Models
This addresses a security vulnerability in LLMs for users relying on thinking modes, though it is incremental as it builds on existing jailbreak research.
The paper finds that large language models (LLMs) in thinking mode are more vulnerable to jailbreak attacks, with higher success rates on benchmarks like AdvBench and HarmBench, and proposes a safe thinking intervention method that significantly reduces these attack rates.
Thinking mode has always been regarded as one of the most valuable modes in LLMs. However, we uncover a surprising and previously overlooked phenomenon: LLMs with thinking mode are more easily broken by Jailbreak attack. We evaluate 9 LLMs on AdvBench and HarmBench and find that the success rate of attacking thinking mode in LLMs is almost higher than that of non-thinking mode. Through large numbers of sample studies, it is found that for educational purposes and excessively long thinking lengths are the characteristics of successfully attacked data, and LLMs also give harmful answers when they mostly know that the questions are harmful. In order to alleviate the above problems, this paper proposes a method of safe thinking intervention for LLMs, which explicitly guides the internal thinking processes of LLMs by adding "specific thinking tokens" of LLMs to the prompt. The results demonstrate that the safe thinking intervention can significantly reduce the attack success rate of LLMs with thinking mode.