Be a Multitude to Itself: A Prompt Evolution Framework for Red Teaming
This work addresses the need for scalable safety testing of LLMs, which is crucial for mitigating harmful content risks before deployment, though it is incremental in automating red teaming processes.
The paper tackles the problem of efficiently generating diverse and effective prompts to test the safety of large language models (LLMs) by proposing RTPE, a scalable evolution framework that surpasses existing methods in attack success rate and diversity, as demonstrated through extensive experiments.
Large Language Models (LLMs) have gained increasing attention for their remarkable capacity, alongside concerns about safety arising from their potential to produce harmful content. Red teaming aims to find prompts that could elicit harmful responses from LLMs, and is essential to discover and mitigate safety risks before real-world deployment. However, manual red teaming is both time-consuming and expensive, rendering it unscalable. In this paper, we propose RTPE, a scalable evolution framework to evolve red teaming prompts across both breadth and depth dimensions, facilitating the automatic generation of numerous high-quality and diverse red teaming prompts. Specifically, in-breadth evolving employs a novel enhanced in-context learning method to create a multitude of quality prompts, whereas in-depth evolving applies customized transformation operations to enhance both content and form of prompts, thereby increasing diversity. Extensive experiments demonstrate that RTPE surpasses existing representative automatic red teaming methods on both attack success rate and diversity. In addition, based on 4,800 red teaming prompts created by RTPE, we further provide a systematic analysis of 8 representative LLMs across 8 sensitive topics.