The Psychosocial Impacts of Generative AI Harms
It addresses the problem of unexamined AI harms in educational settings, particularly for marginalized groups, but is incremental as it extends existing findings on stereotyping.
This paper investigates the psychosocial harms of generative language models by analyzing 150,000 stories from five leading LMs, focusing on stereotyping, erasure, and subordination in student classroom interactions, revealing egregious vignettes that impact marginalized users.
The rapid emergence of generative Language Models (LMs) has led to growing concern about the impacts that their unexamined adoption may have on the social well-being of diverse user groups. Meanwhile, LMs are increasingly being adopted in K-20 schools and one-on-one student settings with minimal investigation of potential harms associated with their deployment. Motivated in part by real-world/everyday use cases (e.g., an AI writing assistant) this paper explores the potential psychosocial harms of stories generated by five leading LMs in response to open-ended prompting. We extend findings of stereotyping harms analyzing a total of 150K 100-word stories related to student classroom interactions. Examining patterns in LM-generated character demographics and representational harms (i.e., erasure, subordination, and stereotyping) we highlight particularly egregious vignettes, illustrating the ways LM-generated outputs may influence the experiences of users with marginalized and minoritized identities, and emphasizing the need for a critical understanding of the psychosocial impacts of generative AI tools when deployed and utilized in diverse social contexts.