Probability of Differentiation Reveals Brittleness of Homogeneity Bias in GPT-4
This research addresses the issue of bias in AI models for developers and researchers, but it is incremental as it builds on prior studies by using a new method to evaluate an existing bias.
The study tackled the problem of homogeneity bias in Large Language Models (LLMs) by directly assessing it in GPT-4's outputs using probability of differentiation, finding that the bias is highly volatile across situation cues and prompts, suggesting past work may reflect encoder model biases rather than LLMs.
Homogeneity bias in Large Language Models (LLMs) refers to their tendency to homogenize the representations of some groups compared to others. Previous studies documenting this bias have predominantly used encoder models, which may have inadvertently introduced biases. To address this limitation, we prompted GPT-4 to generate single word/expression completions associated with 18 situation cues-specific, measurable elements of environments that influence how individuals perceive situations and compared the variability of these completions using probability of differentiation. This approach directly assessed homogeneity bias from the model's outputs, bypassing encoder models. Across five studies, we find that homogeneity bias is highly volatile across situation cues and writing prompts, suggesting that the bias observed in past work may reflect those within encoder models rather than LLMs. Furthermore, we find that homogeneity bias in LLMs is brittle, as even minor and arbitrary changes in prompts can significantly alter the expression of biases. Future work should further explore how variations in syntactic features and topic choices in longer text generations influence homogeneity bias in LLMs.