On the Linearity of Semantic Change: Investigating Meaning Variation via Dynamic Graph Models
This work provides new hypotheses for understanding meaning variation in language, which could benefit computational linguistics and NLP, though it appears incremental in its modeling approach.
The authors tackled the problem of modeling semantic change over time by proposing two dynamic graph models and applying them to corpora in three languages, finding that semantic change is linear in two ways: today's word embeddings can be derived as linear combinations of past embeddings, and self-similarity decays linearly over time.
We consider two graph models of semantic change. The first is a time-series model that relates embedding vectors from one time period to embedding vectors of previous time periods. In the second, we construct one graph for each word: nodes in this graph correspond to time points and edge weights to the similarity of the word's meaning across two time points. We apply our two models to corpora across three different languages. We find that semantic change is linear in two senses. Firstly, today's embedding vectors (= meaning) of words can be derived as linear combinations of embedding vectors of their neighbors in previous time periods. Secondly, self-similarity of words decays linearly in time. We consider both findings as new laws/hypotheses of semantic change.