Neural Morphology Dataset and Models for Multiple Languages, from the Large to the Endangered
This work addresses the challenge of morphological processing for a wide range of languages, particularly endangered ones, by providing scalable neural models and datasets.
The authors tackled the problem of morphological analysis, generation, and lemmatization for morphologically rich languages by automatically extracting large training datasets from finite-state transducers (FSTs) for 22 languages, including 17 endangered ones, and training neural models that align with FST tagsets for use as fallback systems.
We train neural models for morphological analysis, generation and lemmatization for morphologically rich languages. We present a method for automatically extracting substantially large amount of training data from FSTs for 22 languages, out of which 17 are endangered. The neural models follow the same tagset as the FSTs in order to make it possible to use them as fallback systems together with the FSTs. The source code, models and datasets have been released on Zenodo.