Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Models
It addresses safety vulnerabilities in LLMs for developers and users, offering a scalable solution with minimal human intervention, though it appears incremental as it builds on existing red teaming approaches.
The paper tackles the problem of evaluating Large Language Model safety by introducing Jailbreak-Zero, a red teaming method that shifts from example-based to policy-based frameworks, achieving Pareto optimality and significantly higher attack success rates against models like GPT-40 and Claude 3.5 compared to state-of-the-art techniques.
This paper introduces Jailbreak-Zero, a novel red teaming methodology that shifts the paradigm of Large Language Model (LLM) safety evaluation from a constrained example-based approach to a more expansive and effective policy-based framework. By leveraging an attack LLM to generate a high volume of diverse adversarial prompts and then fine-tuning this attack model with a preference dataset, Jailbreak-Zero achieves Pareto optimality across the crucial objectives of policy coverage, attack strategy diversity, and prompt fidelity to real user inputs. The empirical evidence demonstrates the superiority of this method, showcasing significantly higher attack success rates against both open-source and proprietary models like GPT-40 and Claude 3.5 when compared to existing state-of-the-art techniques. Crucially, Jailbreak-Zero accomplishes this while producing human-readable and effective adversarial prompts with minimal need for human intervention, thereby presenting a more scalable and comprehensive solution for identifying and mitigating the safety vulnerabilities of LLMs.