Menta: A Small Language Model for On-Device Mental Health Prediction
This work addresses the need for scalable, privacy-preserving mental health monitoring by enabling on-device deployment, though it is incremental as it builds on existing SLM and fine-tuning techniques.
The authors tackled the problem of deploying mental health prediction models on devices by introducing Menta, a small language model optimized for multi-task classification from social media data, achieving a 15.2% average improvement over non-fine-tuned SLMs and higher accuracy than larger LLMs while being 3.25x smaller and deployable on an iPhone with 3GB RAM.
Mental health conditions affect hundreds of millions globally, yet early detection remains limited. While large language models (LLMs) have shown promise in mental health applications, their size and computational demands hinder practical deployment. Small language models (SLMs) offer a lightweight alternative, but their use for social media--based mental health prediction remains largely underexplored. In this study, we introduce Menta, the first optimized SLM fine-tuned specifically for multi-task mental health prediction from social media data. Menta is jointly trained across six classification tasks using a LoRA-based framework, a cross-dataset strategy, and a balanced accuracy--oriented loss. Evaluated against nine state-of-the-art SLM baselines, Menta achieves an average improvement of 15.2\% across tasks covering depression, stress, and suicidality compared with the best-performing non--fine-tuned SLMs. It also achieves higher accuracy on depression and stress classification tasks compared to 13B-parameter LLMs, while being approximately 3.25x smaller. Moreover, we demonstrate real-time, on-device deployment of Menta on an iPhone 15 Pro Max, requiring only approximately 3GB RAM. Supported by a comprehensive benchmark against existing SLMs and LLMs, Menta highlights the potential for scalable, privacy-preserving mental health monitoring. Code is available at: https://xxue752-nz.github.io/menta-project/