iNLTK: Natural Language Toolkit for Indic Languages
This provides a practical toolkit for NLP in Indic languages, addressing resource scarcity for researchers and developers, though it is incremental as it builds on existing methods.
The authors introduced iNLTK, an open-source NLP library with pre-trained models for 13 Indic languages, which significantly outperformed previous results on text classification tasks and achieved over 95% of prior best performance using less than 10% of training data through data augmentation.
We present iNLTK, an open-source NLP library consisting of pre-trained language models and out-of-the-box support for Data Augmentation, Textual Similarity, Sentence Embeddings, Word Embeddings, Tokenization and Text Generation in 13 Indic Languages. By using pre-trained models from iNLTK for text classification on publicly available datasets, we significantly outperform previously reported results. On these datasets, we also show that by using pre-trained models and data augmentation from iNLTK, we can achieve more than 95% of the previous best performance by using less than 10% of the training data. iNLTK is already being widely used by the community and has 40,000+ downloads, 600+ stars and 100+ forks on GitHub. The library is available at https://github.com/goru001/inltk.