Defending Jailbreak Prompts via In-Context Adversarial Game
This addresses security vulnerabilities in LLMs for users and developers, representing an incremental improvement over traditional static defense methods.
The paper tackles the problem of defending large language models (LLMs) against jailbreak attacks by introducing the In-Context Adversarial Game (ICAG), which dynamically extends knowledge through an iterative adversarial process without fine-tuning, resulting in significantly reduced jailbreak success rates across various attack scenarios and demonstrating transferability to other LLMs.
Large Language Models (LLMs) demonstrate remarkable capabilities across diverse applications. However, concerns regarding their security, particularly the vulnerability to jailbreak attacks, persist. Drawing inspiration from adversarial training in deep learning and LLM agent learning processes, we introduce the In-Context Adversarial Game (ICAG) for defending against jailbreaks without the need for fine-tuning. ICAG leverages agent learning to conduct an adversarial game, aiming to dynamically extend knowledge to defend against jailbreaks. Unlike traditional methods that rely on static datasets, ICAG employs an iterative process to enhance both the defense and attack agents. This continuous improvement process strengthens defenses against newly generated jailbreak prompts. Our empirical studies affirm ICAG's efficacy, where LLMs safeguarded by ICAG exhibit significantly reduced jailbreak success rates across various attack scenarios. Moreover, ICAG demonstrates remarkable transferability to other LLMs, indicating its potential as a versatile defense mechanism.