Morphological Irregularity Correlates with Frequency
This confirms longstanding linguistic proposals about irregularity and frequency, providing evidence for models of linguistic structure, but it is incremental as it builds on prior work with a broader dataset.
The study tackled the relationship between morphological irregularity and word frequency across 28 languages, finding that higher frequency items are more likely to be irregular and irregular items tend to be highly frequent, with the correlation being more robust at the paradigm level.
We present a study of morphological irregularity. Following recent work, we define an information-theoretic measure of irregularity based on the predictability of forms in a language. Using a neural transduction model, we estimate this quantity for the forms in 28 languages. We first present several validatory and exploratory analyses of irregularity. We then show that our analyses provide evidence for a correlation between irregularity and frequency: higher frequency items are more likely to be irregular and irregular items are more likely be highly frequent. To our knowledge, this result is the first of its breadth and confirms longstanding proposals from the linguistics literature. The correlation is more robust when aggregated at the level of whole paradigms--providing support for models of linguistic structure in which inflected forms are unified by abstract underlying stems or lexemes. Code is available at https://github.com/shijie-wu/neural-transducer.