AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs
This addresses the problem of efficient and scalable jailbreaking attacks for LLM security, though it is incremental as it builds on existing adversarial prompting methods.
The paper tackles the vulnerability of LLMs to jailbreaking attacks by introducing AdvPrompter, a method that uses an LLM to generate human-readable adversarial prompts in seconds, achieving competitive results on benchmarks like AdvBench and HarmBench.
Large Language Models (LLMs) are vulnerable to jailbreaking attacks that lead to generation of inappropriate or harmful content. Manual red-teaming requires a time-consuming search for adversarial prompts, whereas automatic adversarial prompt generation often leads to semantically meaningless attacks that do not scale well. In this paper, we present a novel method that uses another LLM, called AdvPrompter, to generate human-readable adversarial prompts in seconds. AdvPrompter, which is trained using an alternating optimization algorithm, generates suffixes that veil the input instruction without changing its meaning, such that the TargetLLM is lured to give a harmful response. Experimental results on popular open source TargetLLMs show highly competitive results on the AdvBench and HarmBench datasets, that also transfer to closed-source black-box LLMs. We also show that training on adversarial suffixes generated by AdvPrompter is a promising strategy for improving the robustness of LLMs to jailbreaking attacks.