CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection
This work addresses vaccine hesitancy detection for public health monitoring in Dutch-speaking regions, but it is incremental as it adapts an existing model to a new domain.
The authors tackled the problem of detecting vaccine hesitancy and arguments in Dutch social media by adapting a pre-trained language model to COVID-19 tweets, resulting in statistically significant gains over the baseline model in binary hesitancy detection and substantial improvements in argument detection across classes.
We present CoNTACT: a Dutch language model adapted to the domain of COVID-19 tweets. The model was developed by continuing the pre-training phase of RobBERT (Delobelle, 2020) by using 2.8M Dutch COVID-19 related tweets posted in 2021. In order to test the performance of the model and compare it to RobBERT, the two models were tested on two tasks: (1) binary vaccine hesitancy detection and (2) detection of arguments for vaccine hesitancy. For both tasks, not only Twitter but also Facebook data was used to show cross-genre performance. In our experiments, CoNTACT showed statistically significant gains over RobBERT in all experiments for task 1. For task 2, we observed substantial improvements in virtually all classes in all experiments. An error analysis indicated that the domain adaptation yielded better representations of domain-specific terminology, causing CoNTACT to make more accurate classification decisions.