Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM
This addresses the problem of security risks in real-world LLM applications for users and developers, offering a generalizable solution, though it is incremental as it builds on existing defensive strategies.
The paper tackles the vulnerability of large language models (LLMs) to jailbreak attacks by proposing GuidelineLLM, a defensive paradigm that identifies harmful queries and provides guidelines to LLMs, reducing the attack success rate by an average of 34.17% without requiring safety fine-tuning of the LLMs.
Despite being empowered with alignment mechanisms, large language models (LLMs) are increasingly vulnerable to emerging jailbreak attacks that can compromise their alignment mechanisms. This vulnerability poses significant risks to real-world applications. Existing work faces challenges in both training efficiency and generalization capabilities (i.e., Reinforcement Learning from Human Feedback and Red-Teaming). Developing effective strategies to enable LLMs to resist continuously evolving jailbreak attempts represents a significant challenge. To address this challenge, we propose a novel defensive paradigm called GuidelineLLM, which assists LLMs in recognizing queries that may have harmful content. Before LLMs respond to a query, GuidelineLLM first identifies potential risks associated with the query, summarizes these risks into guideline suggestions, and then feeds these guidelines to the responding LLMs. Importantly, our approach eliminates the necessity for additional safety fine-tuning of the LLMs themselves; only the GuidelineLLM requires fine-tuning. This characteristic enhances the general applicability of GuidelineLLM across various LLMs. Experimental results demonstrate that GuidelineLLM can significantly reduce the attack success rate (ASR) against LLM (an average reduction of 34.17\% ASR) while maintaining the usefulness of LLM in handling benign queries. The code is available at https://github.com/sqzhang-lazy/GuidelineLLM.