Inclusivity in Large Language Models: Personality Traits and Gender Bias in Scientific Abstracts
This addresses inclusivity and bias issues in LLMs for academic writing, which is an incremental contribution to existing research on stereotypes in AI outputs.
The study evaluated three large language models (Claude 3 Opus, Mistral AI Large, Gemini 1.5 Flash) for gender bias in scientific abstract generation, finding that while they produce human-like text, variations in stylistic features indicate significant gender biases.
Large language models (LLMs) are increasingly utilized to assist in scientific and academic writing, helping authors enhance the coherence of their articles. Previous studies have highlighted stereotypes and biases present in LLM outputs, emphasizing the need to evaluate these models for their alignment with human narrative styles and potential gender biases. In this study, we assess the alignment of three prominent LLMs - Claude 3 Opus, Mistral AI Large, and Gemini 1.5 Flash - by analyzing their performance on benchmark text-generation tasks for scientific abstracts. We employ the Linguistic Inquiry and Word Count (LIWC) framework to extract lexical, psychological, and social features from the generated texts. Our findings indicate that, while these models generally produce text closely resembling human authored content, variations in stylistic features suggest significant gender biases. This research highlights the importance of developing LLMs that maintain a diversity of writing styles to promote inclusivity in academic discourse.