Identifying trends in word frequency dynamics
This work addresses short-term word frequency dynamics for linguists and computational social scientists, but it is incremental as it builds on prior findings about word niche.
The authors tackled the problem of distinguishing persistent from temporary increases in word frequency by introducing a model based on word niche, using large datasets from online discussions and digitized books. The model revealed a strong relationship between changes in word dissemination and frequency, with implications for language evolution.
The word-stock of a language is a complex dynamical system in which words can be created, evolve, and become extinct. Even more dynamic are the short-term fluctuations in word usage by individuals in a population. Building on the recent demonstration that word niche is a strong determinant of future rise or fall in word frequency, here we introduce a model that allows us to distinguish persistent from temporary increases in frequency. Our model is illustrated using a 10^8-word database from an online discussion group and a 10^11-word collection of digitized books. The model reveals a strong relation between changes in word dissemination and changes in frequency. Aside from their implications for short-term word frequency dynamics, these observations are potentially important for language evolution as new words must survive in the short term in order to survive in the long term.