Enhancing Crisis-Related Tweet Classification with Entity-Masked Language Modeling and Multi-Task Learning
This work addresses the challenge of improving classification accuracy for crisis management on social media, though it is incremental as it builds on existing multi-task learning and masking techniques.
The paper tackled the problem of event-related biases and imbalanced labels in crisis-related tweet classification by proposing a combination of entity-masked language modeling and hierarchical multi-label classification, resulting in an absolute F1-score gain of up to 10% for actionable information types on the TREC-IS dataset.
Social media has become an important information source for crisis management and provides quick access to ongoing developments and critical information. However, classification models suffer from event-related biases and highly imbalanced label distributions which still poses a challenging task. To address these challenges, we propose a combination of entity-masked language modeling and hierarchical multi-label classification as a multi-task learning problem. We evaluate our method on tweets from the TREC-IS dataset and show an absolute performance gain w.r.t. F1-score of up to 10% for actionable information types. Moreover, we found that entity-masking reduces the effect of overfitting to in-domain events and enables improvements in cross-event generalization.