Bilingual Word Level Language Identification for Omotic Languages
This work addresses language identification for specific Omotic languages in Ethiopia, which is incremental as it applies existing methods to a new language pair.
The paper tackled bilingual language identification for the closely related Wolaita and Gofa languages in Ethiopia, achieving an F1 score of 0.72 on the test set using a combination of BERT and LSTM.
Language identification is the task of determining the languages for a given text. In many real world scenarios, text may contain more than one language, particularly in multilingual communities. Bilingual Language Identification (BLID) is the task of identifying and distinguishing between two languages in a given text. This paper presents BLID for languages spoken in the southern part of Ethiopia, namely Wolaita and Gofa. The presence of words similarities and differences between the two languages makes the language identification task challenging. To overcome this challenge, we employed various experiments on various approaches. Then, the combination of the BERT based pretrained language model and LSTM approach performed better, with an F1 score of 0.72 on the test set. As a result, the work will be effective in tackling unwanted social media issues and providing a foundation for further research in this area.