FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data
For researchers and practitioners in mental health AI, this work empirically demonstrates the trade-offs of privacy-preserving techniques, highlighting limitations of differential privacy for sparse linguistic markers.
The paper evaluates federated learning (FL) and differentially private FL for depression and suicide detection from social media, finding that FL achieves near-centralized performance (F1 83.16 vs 85.63) but differential privacy causes severe degradation (up to 27.01 F1 drop) even at low noise levels.
Social media text data are often used to train Machine Learning (ML) models to identify users exhibiting high-risk mental health behaviors. However, sharing this sensitive data poses privacy risks and limits the growth of benchmark datasets. We comprehensively evaluate whether privacy-preserving ML techniques can enable safer data sharing while preserving performance. Specifically, we apply federated learning (FL) and Differentially Private FL for two widely-studied mental health prediction tasks: depression detection on X (Twitter) and suicide crisis detection on Reddit. We simulate realistic data-sharing scenarios by treating each user as a client in a non-IID setting, evaluating across different client fractions, aggregation strategies, and privacy budgets. While FL achieves comparable performance to centralized training (centralized F1 = 85.63; best FL model F1 = 83.16) on depression identification, we find that Differentially Private FL has a large performance-privacy trade-off (up to F1 = 27.01 drop) even with low levels of noise (epsilon = 50). This is due to the distortion of highly informative yet sparse mental health linguistic markers related to mental health, like health topics and emotion words. This research empirically demonstrates the potential and limitations of current privacy preservation techniques for mental health inference tasks.