Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling
This work addresses pharmacovigilance for drug regulators, the pharmaceutical industry, and the public, but it is incremental as it builds on existing BERT models and shared task benchmarks.
The paper tackled the problem of detecting adverse drug reactions (ADRs) from Twitter tweets by developing a deep learning model, achieving state-of-the-art performance with an F1-score of 0.6681 and recall of 0.7700.
The automation of adverse drug reaction (ADR) detection in social media would revolutionize the practice of pharmacovigilance, supporting drug regulators, the pharmaceutical industry and the general public in ensuring the safety of the drugs prescribed in daily practice. Following from the published proceedings of the Social Media Mining for Health (SMM4H) Applications Workshop & Shared Task in August 2019, we aimed to develop a deep learning model to classify ADRs within Twitter tweets that contain drug mentions. Our approach involved fine-tuning $BERT_{LARGE}$ and two domain-specific BERT implementations, $BioBERT$ and $Bio + clinicalBERT$, applying a domain-specific preprocessor, and developing a max-prediction ensembling approach. Our final model resulted in state-of-the-art performance on both $F_1$-score (0.6681) and recall (0.7700) outperforming all models submitted in SMM4H 2019 and during post-evaluation to date.