How Prevalent is Gender Bias in ChatGPT? -- Exploring German and English ChatGPT Responses
This study highlights a critical issue for non-IT users relying on ChatGPT, as it reveals inherent biases that could lead to misinformation or unfair outcomes, though it is incremental in building on existing bias research in LLMs.
The paper systematically analyzed gender bias in ChatGPT's responses across English and German, finding that the model exhibits gender biases and inconsistencies when prompted from different gender perspectives, which users must check for.
With the introduction of ChatGPT, OpenAI made large language models (LLM) accessible to users with limited IT expertise. However, users with no background in natural language processing (NLP) might lack a proper understanding of LLMs. Thus the awareness of their inherent limitations, and therefore will take the systems' output at face value. In this paper, we systematically analyse prompts and the generated responses to identify possible problematic issues with a special focus on gender biases, which users need to be aware of when processing the system's output. We explore how ChatGPT reacts in English and German if prompted to answer from a female, male, or neutral perspective. In an in-depth investigation, we examine selected prompts and analyse to what extent responses differ if the system is prompted several times in an identical way. On this basis, we show that ChatGPT is indeed useful for helping non-IT users draft texts for their daily work. However, it is absolutely crucial to thoroughly check the system's responses for biases as well as for syntactic and grammatical mistakes.