L3Cube-MahaNLP: Marathi Natural Language Processing Datasets, Models, and Library
This addresses the problem of limited NLP tools for Marathi speakers and developers, though it is incremental as it applies existing methods to a new language.
The paper tackles the lack of NLP resources for Marathi by creating datasets and models for tasks like sentiment analysis, named entity recognition, and hate speech detection, resulting in the release of MahaCorpus, MahaSent, MahaNER, and MahaHate datasets along with fine-tuned MahaBERT models.
Despite being the third most popular language in India, the Marathi language lacks useful NLP resources. Moreover, popular NLP libraries do not have support for the Marathi language. With L3Cube-MahaNLP, we aim to build resources and a library for Marathi natural language processing. We present datasets and transformer models for supervised tasks like sentiment analysis, named entity recognition, and hate speech detection. We have also published a monolingual Marathi corpus for unsupervised language modeling tasks. Overall we present MahaCorpus, MahaSent, MahaNER, and MahaHate datasets and their corresponding MahaBERT models fine-tuned on these datasets. We aim to move ahead of benchmark datasets and prepare useful resources for Marathi. The resources are available at https://github.com/l3cube-pune/MarathiNLP.