Enhancing Jailbreak Attacks with Diversity Guidance
This work addresses security concerns for LLM users by enhancing jailbreak attacks, though it is incremental as it builds on existing methods to overcome computational limitations.
The paper tackled the vulnerability of large language models (LLMs) to jailbreak attacks by proposing DPP-based Stochastic Trigger Searching (DSTS), a new optimization algorithm that incorporates diversity guidance, which improved attack performance and enabled risk boundary computation for LLM safety evaluation.
As large language models(LLMs) become commonplace in practical applications, the security issues of LLMs have attracted societal concerns. Although extensive efforts have been made to safety alignment, LLMs remain vulnerable to jailbreak attacks. We find that redundant computations limit the performance of existing jailbreak attack methods. Therefore, we propose DPP-based Stochastic Trigger Searching (DSTS), a new optimization algorithm for jailbreak attacks. DSTS incorporates diversity guidance through techniques including stochastic gradient search and DPP selection during optimization. Detailed experiments and ablation studies demonstrate the effectiveness of the algorithm. Moreover, we use the proposed algorithm to compute the risk boundaries for different LLMs, providing a new perspective on LLM safety evaluation.