Who are you, ChatGPT? Personality and Demographic Style in LLM-Generated Content
This work addresses the issue of understanding personality-like traits in AI-generated content for researchers and users of LLMs, though it is incremental as it builds on prior findings on automated agents.
The researchers tackled the problem of assessing personality and demographic characteristics in large language models (LLMs) by introducing a data-driven method using automatic classifiers on Reddit-derived responses, finding that LLMs systematically express higher Agreeableness and lower Neuroticism compared to humans, with gendered patterns resembling humans but with reduced variation.
Generative large language models (LLMs) have become central to everyday life, producing human-like text across diverse domains. A growing body of research investigates whether these models also exhibit personality- and demographic-like characteristics in their language. In this work, we introduce a novel, data-driven methodology for assessing LLM personality without relying on self-report questionnaires, applying instead automatic personality and gender classifiers to model replies on open-ended questions collected from Reddit. Comparing six widely used models to human-authored responses, we find that LLMs systematically express higher Agreeableness and lower Neuroticism, reflecting cooperative and stable conversational tendencies. Gendered language patterns in model text broadly resemble those of human writers, though with reduced variation, echoing prior findings on automated agents. We contribute a new dataset of human and model responses, along with large-scale comparative analyses, shedding new light on the topic of personality and demographic patterns of generative AI.