Rethinking Metrics for Lexical Semantic Change Detection
This work addresses the need for better evaluation metrics in LSCD, an incremental improvement for researchers in computational linguistics and historical linguistics.
The paper tackled the problem of lexical semantic change detection (LSCD) by introducing new metrics, Average Minimum Distance (AMD) and Symmetric Average Minimum Distance (SAMD), which often provide more robust performance than existing metrics like Average Pairwise Distance (APD) and cosine distance over word prototypes (PRT), particularly under dimensionality reduction and with non-specialised encoders.
Lexical semantic change detection (LSCD) increasingly relies on contextualised language model embeddings, yet most approaches still quantify change using a small set of semantic change metrics, primarily Average Pairwise Distance (APD) and cosine distance over word prototypes (PRT). We introduce Average Minimum Distance (AMD) and Symmetric Average Minimum Distance (SAMD), new measures that quantify semantic change via local correspondence between word usages across time periods. Across multiple languages, encoder models, and representation spaces, we show that AMD often provides more robust performance, particularly under dimensionality reduction and with non-specialised encoders, while SAMD excels with specialised encoders. We suggest that LSCD may benefit from considering alternative semantic change metrics beyond APD and PRT, with AMD offering a robust option for contextualised embedding-based analysis.