Retrieval-Augmented Multimodal Depression Detection
This addresses depression detection for mental health applications, offering an incremental improvement over existing multimodal methods.
The paper tackled depression detection by proposing a Retrieval-Augmented Generation framework that retrieves emotional content and uses an LLM to generate an Emotion Prompt as an auxiliary modality, achieving state-of-the-art performance with a CCC of 0.593 and MAE of 3.95 on the AVEC 2019 dataset.
Multimodal deep learning has shown promise in depression detection by integrating text, audio, and video signals. Recent work leverages sentiment analysis to enhance emotional understanding, yet suffers from high computational cost, domain mismatch, and static knowledge limitations. To address these issues, we propose a novel Retrieval-Augmented Generation (RAG) framework. Given a depression-related text, our method retrieves semantically relevant emotional content from a sentiment dataset and uses a Large Language Model (LLM) to generate an Emotion Prompt as an auxiliary modality. This prompt enriches emotional representation and improves interpretability. Experiments on the AVEC 2019 dataset show our approach achieves state-of-the-art performance with CCC of 0.593 and MAE of 3.95, surpassing previous transfer learning and multi-task learning baselines.