BiasJailbreak:Analyzing Ethical Biases and Jailbreak Vulnerabilities in Large Language Models
This addresses safety vulnerabilities in LLMs for users and developers by revealing how biases can lead to harmful outputs, though it builds incrementally on existing jailbreak research.
The paper investigates how ethical biases in large language models (LLMs) can be exploited for jailbreak attacks, showing success rate differences of 20% between non-binary and cisgender keywords and 16% between white and black keywords in GPT-4o models. It proposes BiasJailbreak to automatically generate biased keywords for attacks and BiasDefense as an efficient defense method using pre-generation prompts.
Although large language models (LLMs) demonstrate impressive proficiency in various tasks, they present potential safety risks, such as `jailbreaks', where malicious inputs can coerce LLMs into generating harmful content bypassing safety alignments. In this paper, we delve into the ethical biases in LLMs and examine how those biases could be exploited for jailbreaks. Notably, these biases result in a jailbreaking success rate in GPT-4o models that differs by 20\% between non-binary and cisgender keywords and by 16\% between white and black keywords, even when the other parts of the prompts are identical. We introduce the concept of BiasJailbreak, highlighting the inherent risks posed by these safety-induced biases. BiasJailbreak generates biased keywords automatically by asking the target LLM itself, and utilizes the keywords to generate harmful output. Additionally, we propose an efficient defense method BiasDefense, which prevents jailbreak attempts by injecting defense prompts prior to generation. BiasDefense stands as an appealing alternative to Guard Models, such as Llama-Guard, that require additional inference cost after text generation. Our findings emphasize that ethical biases in LLMs can actually lead to generating unsafe output, and suggest a method to make the LLMs more secure and unbiased. To enable further research and improvements, we open-source our code and artifacts of BiasJailbreak, providing the community with tools to better understand and mitigate safety-induced biases in LLMs.