LSCDiscovery: A shared task on semantic change discovery and detection in Spanish
This work addresses the need for semantic change analysis in Spanish for linguists and NLP researchers, but it is incremental as it extends existing frameworks to a new language and task format.
They tackled the problem of semantic change discovery and detection in Spanish by creating the first manually annotated dataset and organizing a shared task, with the best system achieving a Spearman rank correlation of 0.735 for graded change discovery and an F1 score of 0.716 for binary change detection.
We present the first shared task on semantic change discovery and detection in Spanish and create the first dataset of Spanish words manually annotated for semantic change using the DURel framework (Schlechtweg et al., 2018). The task is divided in two phases: 1) Graded Change Discovery, and 2) Binary Change Detection. In addition to introducing a new language the main novelty with respect to the previous tasks consists in predicting and evaluating changes for all vocabulary words in the corpus. Six teams participated in phase 1 and seven teams in phase 2 of the shared task, and the best system obtained a Spearman rank correlation of 0.735 for phase 1 and an F1 score of 0.716 for phase 2. We describe the systems developed by the competing teams, highlighting the techniques that were particularly useful and discuss the limits of these approaches.