Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shifts
This work addresses the problem of NER in social media for researchers and practitioners, but it is incremental as it focuses on dataset creation and temporal analysis without introducing a novel method.
The authors tackled Named Entity Recognition (NER) in noisy and dynamic Twitter data by constructing TweetNER7, a new dataset with 11,382 tweets annotated for seven entity types from 2019 to 2021, and analyzed model performance degradation over time, fine-tuning strategies, and self-labeling methods.
Recent progress in language model pre-training has led to important improvements in Named Entity Recognition (NER). Nonetheless, this progress has been mainly tested in well-formatted documents such as news, Wikipedia, or scientific articles. In social media the landscape is different, in which it adds another layer of complexity due to its noisy and dynamic nature. In this paper, we focus on NER in Twitter, one of the largest social media platforms, and construct a new NER dataset, TweetNER7, which contains seven entity types annotated over 11,382 tweets from September 2019 to August 2021. The dataset was constructed by carefully distributing the tweets over time and taking representative trends as a basis. Along with the dataset, we provide a set of language model baselines and perform an analysis on the language model performance on the task, especially analyzing the impact of different time periods. In particular, we focus on three important temporal aspects in our analysis: short-term degradation of NER models over time, strategies to fine-tune a language model over different periods, and self-labeling as an alternative to lack of recently-labeled data. TweetNER7 is released publicly (https://huggingface.co/datasets/tner/tweetner7) along with the models fine-tuned on it.