Substitution-based Semantic Change Detection using Contextual Embeddings
This work addresses the challenge of measuring semantic change for researchers in computational linguistics, offering an interpretable and efficient solution, though it is incremental as it builds upon existing contextual embedding techniques.
The paper tackled the problem of semantic change detection by introducing a simplified method using contextual embeddings and most probable substitutes for masked terms, achieving superior average performance across frequently cited datasets.
Measuring semantic change has thus far remained a task where methods using contextual embeddings have struggled to improve upon simpler techniques relying only on static word vectors. Moreover, many of the previously proposed approaches suffer from downsides related to scalability and ease of interpretation. We present a simplified approach to measuring semantic change using contextual embeddings, relying only on the most probable substitutes for masked terms. Not only is this approach directly interpretable, it is also far more efficient in terms of storage, achieves superior average performance across the most frequently cited datasets for this task, and allows for more nuanced investigation of change than is possible with static word vectors.