Stem-driven Language Models for Morphologically Rich Languages
This work addresses the challenge of language modeling for morphologically complex languages, which is important for NLP applications in those linguistic domains, though it is incremental in its approach.
The authors tackled the problem of improving neural language models for morphologically rich languages by incorporating stem-level information, achieving significant perplexity gains compared to baseline models in Hindi, Tamil, Kannada, and Finnish.
Neural language models (LMs) have shown to benefit significantly from enhancing word vectors with subword-level information, especially for morphologically rich languages. This has been mainly tackled by providing subword-level information as an input; using subword units in the output layer has been far less explored. In this work, we propose LMs that are cognizant of the underlying stems in each word. We derive stems for words using a simple unsupervised technique for stem identification. We experiment with different architectures involving multi-task learning and mixture models over words and stems. We focus on four morphologically complex languages -- Hindi, Tamil, Kannada and Finnish -- and observe significant perplexity gains with using our stem-driven LMs when compared with other competitive baseline models.