Investigating Machine Learning Methods for Language and Dialect Identification of Cuneiform Texts
This work addresses the challenge of language and dialect identification for cuneiform texts, which is important for historical linguistics and digital humanities, but it is incremental as it applies existing methods to a specific dataset.
The paper tackled the problem of identifying seven languages and dialects in cuneiform texts, such as Sumerian and Akkadian dialects, by participating in the VarDial 2019 task, achieving a macro-averaged F1-score of 72.10% with an ensemble of Support Vector Machines and a naive Bayes classifier using character-level features.
Identification of the languages written using cuneiform symbols is a difficult task due to the lack of resources and the problem of tokenization. The Cuneiform Language Identification task in VarDial 2019 addresses the problem of identifying seven languages and dialects written in cuneiform; Sumerian and six dialects of Akkadian language: Old Babylonian, Middle Babylonian Peripheral, Standard Babylonian, Neo-Babylonian, Late Babylonian, and Neo-Assyrian. This paper describes the approaches taken by SharifCL team to this problem in VarDial 2019. The best result belongs to an ensemble of Support Vector Machines and a naive Bayes classifier, both working on character-level features, with macro-averaged F1-score of 72.10%.