FLIRT: Feedback Loop In-context Red Teaming
This addresses safety concerns for public users of generative models by exposing vulnerabilities, though it is incremental as it builds on existing red teaming and in-context learning methods.
The paper tackles the problem of testing vulnerabilities in generative models by proposing an automatic red teaming framework that uses in-context learning in a feedback loop to trigger unsafe content generation, demonstrating that even enhanced Stable Diffusion models are vulnerable to adversarial prompts.
Warning: this paper contains content that may be inappropriate or offensive. As generative models become available for public use in various applications, testing and analyzing vulnerabilities of these models has become a priority. In this work, we propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities against unsafe and inappropriate content generation. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation. In particular, taking text-to-image models as target models, we explore different feedback mechanisms to automatically learn effective and diverse adversarial prompts. Our experiments demonstrate that even with enhanced safety features, Stable Diffusion (SD) models are vulnerable to our adversarial prompts, raising concerns on their robustness in practical uses. Furthermore, we demonstrate that the proposed framework is effective for red teaming text-to-text models.