BanglaLlama: LLaMA for Bangla Language
This work addresses the lack of high-quality models for Bangla, a widely spoken low-resource language, by providing datasets and models to improve language processing for its approximately 300 million speakers.
The paper tackled the problem of low-resource Bangla language processing by introducing two translated instruction datasets totaling 224k samples and developing BanglaLlama, a family of Bangla-specific LLMs, which achieved new standard baseline results on multiple benchmarks.
Bangla is a language spoken by approximately 240 million native speakers and around 300 million people worldwide. Despite being the 5th largest spoken language in the world, Bangla is still a "low-resource" language, and existing pretrained language models often struggle to perform well on Bangla Language Processing (BLP) tasks. This paper addresses this gap by: (1) introducing two high-quality translated Bangla-instruction datasets totaling 224k samples - Bangla-Orca (172k) and Bangla-Alpaca (52k); and (2) leveraging these datasets to develop BanglaLlama, an open-source family of Bangla-specific LLMs, consisting of five base and instruct variants. We present our methodology, two large datasets, and comprehensive benchmarking results showcasing the effectiveness of our dataset and model on multiple benchmarks. We believe our proposed datasets and models will serve as the new standard baseline for future research focused on this widely spoken yet "low-resource" language.