Uralic Language Identification (ULI) 2020 shared task dataset and the Wanca 2017 corpus
This work addresses the problem of language identification for rare Uralic languages, but it is incremental as it focuses on dataset creation and baseline results.
The paper introduces the Wanca 2017 corpus and the ULI 2020 dataset for rare Uralic languages, and provides baseline language identification experiments.
This article introduces the Wanca 2017 corpus of texts crawled from the internet from which the sentences in rare Uralic languages for the use of the Uralic Language Identification (ULI) 2020 shared task were collected. We describe the ULI dataset and how it was constructed using the Wanca 2017 corpus and texts in different languages from the Leipzig corpora collection. We also provide baseline language identification experiments conducted using the ULI 2020 dataset.