Fusing Visual, Textual and Connectivity Clues for Studying Mental Health
This work addresses mental health monitoring on social media, offering incremental improvements for researchers and potential applications in demographic-aware health interventions.
The paper tackled the problem of identifying depressed individuals on social media by developing a multimodal framework that fuses visual, textual, and user interaction features, improving the average F1-Score by 5% over existing state-of-the-art approaches.
With ubiquity of social media platforms, millions of people are sharing their online persona by expressing their thoughts, moods, emotions, feelings, and even their daily struggles with mental health issues voluntarily and publicly on social media. Unlike the most existing efforts which study depression by analyzing textual content, we examine and exploit multimodal big data to discern depressive behavior using a wide variety of features including individual-level demographics. By developing a multimodal framework and employing statistical techniques for fusing heterogeneous sets of features obtained by processing visual, textual and user interaction data, we significantly enhance the current state-of-the-art approaches for identifying depressed individuals on Twitter (improving the average F1-Score by 5 percent) as well as facilitate demographic inference from social media for broader applications. Besides providing insights into the relationship between demographics and mental health, our research assists in the design of a new breed of demographic-aware health interventions.