Detection of depression on social networks using transformers and ensembles
This work addresses mental health monitoring by detecting depression signs in social media posts, but it is incremental as it applies existing transformer methods to this domain.
The paper tackled automatic depression detection from social media posts by building classifiers using pre-trained language models like BERT and RoBERTa, along with ensembles, and found that transformer ensembles improved performance over single models on Reddit and Twitter datasets.
As the impact of technology on our lives is increasing, we witness increased use of social media that became an essential tool not only for communication but also for sharing information with community about our thoughts and feelings. This can be observed also for people with mental health disorders such as depression where they use social media for expressing their thoughts and asking for help. This opens a possibility to automatically process social media posts and detect signs of depression. We build several large pre-trained language model based classifiers for depression detection from social media posts. Besides fine-tuning BERT, RoBERTA, BERTweet, and mentalBERT were also construct two types of ensembles. We analyze the performance of our models on two data sets of posts from social platforms Reddit and Twitter, and investigate also the performance of transfer learning across the two data sets. The results show that transformer ensembles improve over the single transformer-based classifiers.