Improving Disease Detection from Social Media Text via Self-Augmentation and Contrastive Learning
This work addresses the challenge of refining language models for disease detection from social media, which is important for applications like public health monitoring, but it appears incremental as it builds on existing contrastive learning and augmentation techniques.
The paper tackled the problem of improving disease detection from social media text by proposing a method that integrates contrastive learning with language modeling and self-augmentation, resulting in up to a 2.48% increase in F1-score over baselines and 2.1% over state-of-the-art methods.
Detecting diseases from social media has diverse applications, such as public health monitoring and disease spread detection. While language models (LMs) have shown promising performance in this domain, there remains ongoing research aimed at refining their discriminating representations. In this paper, we propose a novel method that integrates Contrastive Learning (CL) with language modeling to address this challenge. Our approach introduces a self-augmentation method, wherein hidden representations of the model are augmented with their own representations. This method comprises two branches: the first branch, a traditional LM, learns features specific to the given data, while the second branch incorporates augmented representations from the first branch to encourage generalization. CL further refines these representations by pulling pairs of original and augmented versions closer while pushing other samples away. We evaluate our method on three NLP datasets encompassing binary, multi-label, and multi-class classification tasks involving social media posts related to various diseases. Our approach demonstrates notable improvements over traditional fine-tuning methods, achieving up to a 2.48% increase in F1-score compared to baseline approaches and a 2.1% enhancement over state-of-the-art methods.