LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment
This work addresses mental health assessment accessibility, particularly in resource-limited settings, by demonstrating LLMs' potential as efficient tools, though it is incremental as it builds on existing prompting methods for a specific clinical scale.
The study tackled automating depression severity assessment by introducing LlaMADRS, a framework using open-source LLMs with zero-shot prompting to score clinical interviews based on the MADRS, achieving near-human level agreement with clinician assessments, such as the Qwen 2.5--72b model closely approaching human rater ICCs on 236 real-world interviews.
This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment using the Montgomery-Asberg Depression Rating Scale (MADRS). We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews. Our approach, tested on 236 real-world interviews from the Context-Adaptive Multimodal Informatics (CAMI) dataset, demonstrates strong correlations with clinician assessments. The Qwen 2.5--72b model achieves near-human level agreement across most MADRS items, with Intraclass Correlation Coefficients (ICC) closely approaching those between human raters. We provide a comprehensive analysis of model performance across different MADRS items, highlighting strengths and current limitations. Our findings suggest that LLMs, with appropriate prompting, can serve as efficient tools for mental health assessment, potentially increasing accessibility in resource-limited settings. However, challenges remain, particularly in assessing symptoms that rely on non-verbal cues, underscoring the need for multimodal approaches in future work.