Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks
This addresses the vulnerability of LLMs to adversarial attacks, providing a robust defense mechanism for AI safety applications.
The paper tackles the problem of defending large language models against jailbreaking attacks by proposing Robust Prompt Optimization (RPO), which reduces the attack success rate to 6% on GPT-4 and 0% on Llama-2 on JailbreakBench.
Despite advances in AI alignment, large language models (LLMs) remain vulnerable to adversarial attacks or jailbreaking, in which adversaries can modify prompts to induce unwanted behavior. While some defenses have been proposed, they have not been adapted to newly proposed attacks and more challenging threat models. To address this, we propose an optimization-based objective for defending LLMs against jailbreaking attacks and an algorithm, Robust Prompt Optimization (RPO) to create robust system-level defenses. Our approach directly incorporates the adversary into the defensive objective and optimizes a lightweight and transferable suffix, enabling RPO to adapt to worst-case adaptive attacks. Our theoretical and experimental results show improved robustness to both jailbreaks seen during optimization and unknown jailbreaks, reducing the attack success rate (ASR) on GPT-4 to 6% and Llama-2 to 0% on JailbreakBench, setting the state-of-the-art. Code can be found at https://github.com/lapisrocks/rpo