Protected group bias and stereotypes in Large Language Models
It highlights fairness issues in LLMs that could harm minoritized groups, though it is incremental as it builds on existing bias studies.
This paper investigated bias in a publicly available Large Language Model (LLM) by analyzing over 10,000 sentence completions related to protected groups like gender, sexuality, religion, and race, finding that the model amplifies societal biases, particularly in gender and sexuality, and exhibits Western bias.
As modern Large Language Models (LLMs) shatter many state-of-the-art benchmarks in a variety of domains, this paper investigates their behavior in the domains of ethics and fairness, focusing on protected group bias. We conduct a two-part study: first, we solicit sentence continuations describing the occupations of individuals from different protected groups, including gender, sexuality, religion, and race. Second, we have the model generate stories about individuals who hold different types of occupations. We collect >10k sentence completions made by a publicly available LLM, which we subject to human annotation. We find bias across minoritized groups, but in particular in the domains of gender and sexuality, as well as Western bias, in model generations. The model not only reflects societal biases, but appears to amplify them. The model is additionally overly cautious in replies to queries relating to minoritized groups, providing responses that strongly emphasize diversity and equity to an extent that other group characteristics are overshadowed. This suggests that artificially constraining potentially harmful outputs may itself lead to harm, and should be applied in a careful and controlled manner.