Jailbreaking Safeguarded Text-to-Image Models via Large Language Models
This addresses a security vulnerability in text-to-image models for users and developers, but it is incremental as it builds on existing jailbreak techniques.
The paper tackles the problem of bypassing safety guardrails in text-to-image models to generate harmful content, proposing PromptTune, a method using a fine-tuned large language model to efficiently generate adversarial prompts, which effectively bypasses safety guardrails on three datasets and outperforms existing no-box attacks.
Text-to-Image models may generate harmful content, such as pornographic images, particularly when unsafe prompts are submitted. To address this issue, safety filters are often added on top of text-to-image models, or the models themselves are aligned to reduce harmful outputs. However, these defenses remain vulnerable when an attacker strategically designs adversarial prompts to bypass these safety guardrails. In this work, we propose PromptTune, a method to jailbreak text-to-image models with safety guardrails using a fine-tuned large language model. Unlike other query-based jailbreak attacks that require repeated queries to the target model, our attack generates adversarial prompts efficiently after fine-tuning our AttackLLM. We evaluate our method on three datasets of unsafe prompts and against five safety guardrails. Our results demonstrate that our approach effectively bypasses safety guardrails, outperforms existing no-box attacks, and also facilitates other query-based attacks.