STAR-Teaming: A Strategy-Response Multiplex Network Approach to Automated LLM Red Teaming
This work addresses the need for efficient and interpretable automated red teaming to identify vulnerabilities in LLMs, offering a practical tool for safety evaluation.
STAR-Teaming introduces a black-box framework for automated red teaming of LLMs, using a Multi-Agent System with a Strategy-Response Multiplex Network to generate jailbreak prompts. It achieves higher attack success rates at lower computational cost compared to existing methods.
While Large Language Models (LLMs) are widely used, they remain susceptible to jailbreak prompts that can elicit harmful or inappropriate responses. This paper introduces STAR-Teaming, a novel black-box framework for automated red teaming that effectively generates such prompts. STAR-Teaming integrates a Multi-Agent System (MAS) with a Strategy-Response Multiplex Network and employs network-driven optimization to sample effective attack strategies. This network-based approach recasts the intractable high-dimensional embedding space into a tractable structure, yielding two key advantages: it enhances the interpretability of the LLM's strategic vulnerabilities, and it streamlines the search for effective strategies by organizing the search space into semantic communities, thereby preventing redundant exploration. Empirical results demonstrate that STAR-Teaming significantly surpasses existing methods, achieving a higher attack success rate (ASR) at a lower computational cost. Extensive experiments validate the effectiveness and explainability of the Multiplex Network. The code is available at https://github.com/selectstar-ai/STAR-Teaming-paper.