Debiased Large Language Models Still Associate Muslims with Uniquely Violent Acts
This highlights a critical issue for AI ethics and fairness, showing that debiasing efforts are insufficient to address higher-order biases in large language models, which can perpetuate harmful stereotypes.
The study investigated bias in GPT-3, finding that while recent debiased versions show minimal bias in direct prompts, using names associated with religions significantly increases violent completions, revealing a stronger second-order bias against Muslims, with content analysis uncovering offensive themes.
Recent work demonstrates a bias in the GPT-3 model towards generating violent text completions when prompted about Muslims, compared with Christians and Hindus. Two pre-registered replication attempts, one exact and one approximate, found only the weakest bias in the more recent Instruct Series version of GPT-3, fine-tuned to eliminate biased and toxic outputs. Few violent completions were observed. Additional pre-registered experiments, however, showed that using common names associated with the religions in prompts yields a highly significant increase in violent completions, also revealing a stronger second-order bias against Muslims. Names of Muslim celebrities from non-violent domains resulted in relatively fewer violent completions, suggesting that access to individualized information can steer the model away from using stereotypes. Nonetheless, content analysis revealed religion-specific violent themes containing highly offensive ideas regardless of prompt format. Our results show the need for additional debiasing of large language models to address higher-order schemas and associations.