Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Support
This work addresses the need for responsible AI in mental health support, though it appears incremental by applying existing RAG methods to a new domain.
The paper tackled the challenge of developing a safe and effective mental health support chatbot using a retrieval-augmented generation (RAG) framework, achieving a BERT Score of 0.898 and satisfactory results across multiple evaluation metrics.
The emergence of large language models (LLMs) has unlocked boundless possibilities, along with significant challenges. In response, we developed a mental health support chatbot designed to augment professional healthcare, with a strong emphasis on safe and meaningful application. Our approach involved rigorous evaluation, covering accuracy, empathy, trustworthiness, privacy, and bias. We employed a retrieval-augmented generation (RAG) framework, integrated prompt engineering, and fine-tuned a pre-trained model on novel datasets. The resulting system, Mentalic Net Conversational AI, achieved a BERT Score of 0.898, with other evaluation metrics falling within satisfactory ranges. We advocate for a human-in-the-loop approach and a long-term, responsible strategy in developing such transformative technologies, recognizing both their potential to change lives and the risks they may pose if not carefully managed.