Online Learning Defense against Iterative Jailbreak Attacks via Prompt Optimization
This addresses a critical security vulnerability in LLMs for users and developers by providing a proactive defense against iterative jailbreak attacks, though it is incremental as it builds on existing reinforcement learning and defense methods.
The paper tackles the problem of iterative jailbreak attacks on large language models (LLMs) by proposing an online learning defense framework that dynamically updates strategies, significantly outperforming five existing defenses against five attack methods and also improving response quality for harmless tasks.
Iterative jailbreak methods that repeatedly rewrite and input prompts into large language models (LLMs) to induce harmful outputs -- using the model's previous responses to guide each new iteration -- have been found to be a highly effective attack strategy. Despite being an effective attack strategy against LLMs and their safety mechanisms, existing defenses do not proactively disrupt this dynamic trial-and-error cycle. In this study, we propose a novel framework that dynamically updates its defense strategy through online learning in response to each new prompt from iterative jailbreak methods. Leveraging the distinctions between harmful jailbreak-generated prompts and typical harmless prompts, we introduce a reinforcement learning-based approach that optimizes prompts to ensure appropriate responses for harmless tasks while explicitly rejecting harmful prompts. Additionally, to curb overfitting to the narrow band of partial input rewrites explored during an attack, we introduce Past-Direction Gradient Damping (PDGD). Experiments conducted on three LLMs show that our approach significantly outperforms five existing defense methods against five iterative jailbreak methods. Moreover, our results indicate that our prompt optimization strategy simultaneously enhances response quality for harmless tasks.