Transformer-based Model for Word Level Language Identification in Code-mixed Kannada-English Texts
This addresses the problem of language identification in social media code-mixed texts for NLP researchers, but it is incremental as it applies an existing method to a specific dataset.
The paper tackled word-level language identification in code-mixed Kannada-English texts by proposing a Transformer-based model, achieving a weighted F1-score of 0.84 and a macro F1-score of 0.61 on the CoLI-Kenglish dataset.
Using code-mixed data in natural language processing (NLP) research currently gets a lot of attention. Language identification of social media code-mixed text has been an interesting problem of study in recent years due to the advancement and influences of social media in communication. This paper presents the Instituto Politécnico Nacional, Centro de Investigación en Computación (CIC) team's system description paper for the CoLI-Kanglish shared task at ICON2022. In this paper, we propose the use of a Transformer based model for word-level language identification in code-mixed Kannada English texts. The proposed model on the CoLI-Kenglish dataset achieves a weighted F1-score of 0.84 and a macro F1-score of 0.61.