Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation
This addresses security vulnerabilities in LLMs for developers and users by exposing a new attack vector, though it is incremental in building on existing jailbreak research.
The paper tackles the problem of jailbreak attacks on large language models by introducing Dialogue Injection Attack (DIA), a novel paradigm that manipulates dialogue history to enhance attack success rates, achieving state-of-the-art results on models like Llama-3.1 and GPT-4o and bypassing 5 defense mechanisms.
Large language models (LLMs) have demonstrated significant utility in a wide range of applications; however, their deployment is plagued by security vulnerabilities, notably jailbreak attacks. These attacks manipulate LLMs to generate harmful or unethical content by crafting adversarial prompts. While much of the current research on jailbreak attacks has focused on single-turn interactions, it has largely overlooked the impact of historical dialogues on model behavior. In this paper, we introduce a novel jailbreak paradigm, Dialogue Injection Attack (DIA), which leverages the dialogue history to enhance the success rates of such attacks. DIA operates in a black-box setting, requiring only access to the chat API or knowledge of the LLM's chat template. We propose two methods for constructing adversarial historical dialogues: one adapts gray-box prefilling attacks, and the other exploits deferred responses. Our experiments show that DIA achieves state-of-the-art attack success rates on recent LLMs, including Llama-3.1 and GPT-4o. Additionally, we demonstrate that DIA can bypass 5 different defense mechanisms, highlighting its robustness and effectiveness.