A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks
This addresses a security threat for LLM deployment by revealing why existing defenses fail against multi-turn attacks, though it is incremental in analyzing a specific attack method.
The paper tackled the problem of multi-turn jailbreak attacks on safety-aligned LLMs, finding that these attacks keep model representations in a benign region, making defenses like circuit breakers ineffective.
Recent research has demonstrated that state-of-the-art LLMs and defenses remain susceptible to multi-turn jailbreak attacks. These attacks require only closed-box model access and are often easy to perform manually, posing a significant threat to the safe and secure deployment of LLM-based systems. We study the effectiveness of the Crescendo multi-turn jailbreak at the level of intermediate model representations and find that safety-aligned LMs often represent Crescendo responses as more benign than harmful, especially as the number of conversation turns increases. Our analysis indicates that at each turn, Crescendo prompts tend to keep model outputs in a "benign" region of representation space, effectively tricking the model into fulfilling harmful requests. Further, our results help explain why single-turn jailbreak defenses like circuit breakers are generally ineffective against multi-turn attacks, motivating the development of mitigations that address this generalization gap.