Public Perceptions of Gender Bias in Large Language Models: Cases of ChatGPT and Ernie
This research addresses the problem of gender bias in LLMs for users and policymakers by highlighting cultural influences, but it is incremental as it builds on existing studies of bias in AI.
The study analyzed social media discussions to assess public perceptions of gender bias in ChatGPT (US-based) and Ernie (China-based) large language models, finding that ChatGPT exhibited more implicit bias (e.g., gendered profession associations) while Ernie showed explicit bias (e.g., prioritizing marriage over career for women).
Large language models are quickly gaining momentum, yet are found to demonstrate gender bias in their responses. In this paper, we conducted a content analysis of social media discussions to gauge public perceptions of gender bias in LLMs which are trained in different cultural contexts, i.e., ChatGPT, a US-based LLM, or Ernie, a China-based LLM. People shared both observations of gender bias in their personal use and scientific findings about gender bias in LLMs. A difference between the two LLMs was seen -- ChatGPT was more often found to carry implicit gender bias, e.g., associating men and women with different profession titles, while explicit gender bias was found in Ernie's responses, e.g., overly promoting women's pursuit of marriage over career. Based on the findings, we reflect on the impact of culture on gender bias and propose governance recommendations to regulate gender bias in LLMs.