Using Two Losses and Two Datasets Simultaneously to Improve TempoWiC Accuracy
This work addresses the problem of temporal word sense disambiguation for natural language processing researchers, but it is incremental as it builds on existing methods with minor enhancements.
The paper tackles the challenge of Word Sense Disambiguation (WSD) on the TempoWiC dataset, which focuses on temporal word changes, by using two losses and an additional dataset to train RoBERTa-based models, resulting in a 4.23% improvement over the baseline to achieve 74.56% macro-F1.
WSD (Word Sense Disambiguation) is the task of identifying which sense of a word is meant in a sentence or other segment of text. Researchers have worked on this task (e.g. Pustejovsky, 2002) for years but it's still a challenging one even for SOTA (state-of-the-art) LMs (language models). The new dataset, TempoWiC introduced by Loureiro et al. (2022b) focuses on the fact that words change over time. Their best baseline achieves 70.33% macro-F1. In this work, we use two different losses simultaneously to train RoBERTa-based classification models. We also improve our model by using another similar dataset to generalize better. Our best configuration beats their best baseline by 4.23% and reaches 74.56% macroF1.