Learning dynamic word embeddings with drift regularisation
This work addresses the challenge of unsupervised diachronic word embedding analysis for researchers in computational linguistics, but it is incremental as it builds on existing models.
The paper tackled the problem of tracking word meaning changes over time by applying variants of the Dynamic Bernoulli Embeddings model to analyze word evolution in English and French newspaper corpora, resulting in a pipeline for cross-linguistic word use analysis.
Word usage, meaning and connotation change throughout time. Diachronic word embeddings are used to grasp these changes in an unsupervised way. In this paper, we use variants of the Dynamic Bernoulli Embeddings model to learn dynamic word embeddings, in order to identify notable properties of the model. The comparison is made on the New York Times Annotated Corpus in English and a set of articles from the French newspaper Le Monde covering the same period. This allows us to define a pipeline to analyse the evolution of words use across two languages.