From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents
This work addresses psychological risks for users of AI conversational agents, particularly those with mental health experiences, by providing a detailed taxonomy and actionable insights, though it is incremental in focusing on a specific risk category within existing AI risk research.
The study tackled the under-representation of psychological risks in AI conversational agents by developing a novel taxonomy based on lived experiences, resulting in a framework with 19 AI behaviors, 21 negative psychological impacts, and 15 user contexts, along with design recommendations for safer AI.
Recent gains in popularity of AI conversational agents have led to their increased use for improving productivity and supporting well-being. While previous research has aimed to understand the risks associated with interactions with AI conversational agents, these studies often fall short in capturing the lived experiences of individuals. Additionally, psychological risks have often been presented as a sub-category within broader AI-related risks in past taxonomy works, leading to under-representation of the impact of psychological risks of AI use. To address these challenges, our work presents a novel risk taxonomy focusing on psychological risks of using AI gathered through the lived experiences of individuals. We employed a mixed-method approach, involving a comprehensive survey with 283 people with lived mental health experience and workshops involving experts with lived experience to develop a psychological risk taxonomy. Our taxonomy features 19 AI behaviors, 21 negative psychological impacts, and 15 contexts related to individuals. Additionally, we propose a novel multi-path vignette-based framework for understanding the complex interplay between AI behaviors, psychological impacts, and individual user contexts. Finally, based on the feedback obtained from the workshop sessions, we present design recommendations for developing safer and more robust AI agents. Our work offers an in-depth understanding of the psychological risks associated with AI conversational agents and provides actionable recommendations for policymakers, researchers, and developers.