Prefix Guidance: A Steering Wheel for Large Language Models to Defend Against Jailbreak Attacks
This addresses security vulnerabilities in LLMs for users and developers, representing an incremental improvement over existing defense methods.
The paper tackles the problem of jailbreak attacks on large language models (LLMs) by proposing Prefix Guidance (PG), a plug-and-play defense framework that sets the first few tokens of the model's output to identify harmful prompts, resulting in improved effectiveness across three models and five attack methods while preserving model performance on benchmarks.
In recent years, the rapid development of large language models (LLMs) has achieved remarkable performance across various tasks. However, research indicates that LLMs are vulnerable to jailbreak attacks, where adversaries can induce the generation of harmful content through meticulously crafted prompts. This vulnerability poses significant challenges to the secure use and promotion of LLMs. Existing defense methods offer protection from different perspectives but often suffer from insufficient effectiveness or a significant impact on the model's capabilities. In this paper, we propose a plug-and-play and easy-to-deploy jailbreak defense framework, namely Prefix Guidance (PG), which guides the model to identify harmful prompts by directly setting the first few tokens of the model's output. This approach combines the model's inherent security capabilities with an external classifier to defend against jailbreak attacks. We demonstrate the effectiveness of PG across three models and five attack methods. Compared to baselines, our approach is generally more effective on average. Additionally, results on the Just-Eval benchmark further confirm PG's superiority to preserve the model's performance. our code is available at https://github.com/weiyezhimeng/Prefix-Guidance.