A clinically validated framework for auditing AI chatbot behavior in mental health interactions
This provides a scalable safety evaluation framework for mental-health chatbot interactions, addressing an urgent need for rigorous auditing of consumer AI chatbots.
The paper introduces SIM-VAIL, a clinically validated framework for auditing AI chatbot behavior in mental-health contexts. Across 810 conversations with 9 chatbots and 30 simulated user profiles, concerning behavior was widespread but reduced in newer models, and risk could be reduced by interventions at early escalation points.
Millions of users turn to consumer AI chatbots to discuss emotional, behavioral, and mental-health concerns, creating an urgent need for rigorous and scalable safety evaluations. Here we introduce SIM-VAIL, a clinically validated framework for auditing chatbot behavior in mental-health contexts. SIM-VAIL simulates users with specific psychiatric vulnerabilities and conversational intents, engages them in multi-turn conversations with frontier AI chatbots (including Claude, ChatGPT, Gemini, Grok and Llama models), and scores each exchange across 13 clinically grounded risk dimensions. Across 810 conversations, spanning 9 target chatbots and 30 simulated user profiles, concerning behavior in target chatbots was widespread, albeit reduced in newer models. Concerning behavior varied by user vulnerability and conversational intent, accumulated over turns, and could be reduced by interventions at early escalation points. Risk was highest when otherwise supportive chatbot behaviors reinforced the psychological mechanisms underlying the simulated user's vulnerability, a pattern we term a Vulnerability Amplifying Interaction Loop (VAIL). SIM-VAIL provides a scalable framework for mapping mental health risk across users, chatbots, and conversational trajectories, offering a foundation for targeted safety improvements.