AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression
This work addresses risks for people with depression using AI chatbots, but it is incremental as it builds on existing research to identify values and harms without introducing new methods.
The study investigated the potential harms of AI chatbots in mental health by interviewing 17 individuals with depression using a GPT-4o-based chatbot, revealing key values such as informational support and privacy to inform safer design.
Recent advancements in LLMs enable chatbots to interact with individuals on a range of queries, including sensitive mental health contexts. Despite uncertainties about their effectiveness and reliability, the development of LLMs in these areas is growing, potentially leading to harms. To better identify and mitigate these harms, it is critical to understand how the values of people with lived experiences relate to the harms. In this study, we developed a technology probe, a GPT-4o based chatbot called Zenny, enabling participants to engage with depression self-management scenarios informed by previous research. We used Zenny to interview 17 individuals with lived experiences of depression. Our thematic analysis revealed key values: informational support, emotional support, personalization, privacy, and crisis management. This work explores the relationship between lived experience values, potential harms, and design recommendations for mental health AI chatbots, aiming to enhance self-management support while minimizing risks.