NepaliGPT: A Generative Language Model for the Nepali Language
This addresses the problem of limited NLP resources for the Nepali language, though it is incremental as it applies existing LLM methods to a new language.
The research tackled the lack of a generative language model for the Nepali language by proposing NepaliGPT, which achieved a perplexity of 26.32245, ROUGE-1 score of 0.2604, causal coherence of 81.25%, and causal consistency of 85.41%.
After the release of ChatGPT, Large Language Models (LLMs) have gained huge popularity in recent days and thousands of variants of LLMs have been released. However, there is no generative language model for the Nepali language, due to which other downstream tasks, including fine-tuning, have not been explored yet. To fill this research gap in the Nepali NLP space, this research proposes \textit{NepaliGPT}, a generative large language model tailored specifically for the Nepali language. This research introduces an advanced corpus for the Nepali language collected from several sources, called the Devanagari Corpus. Likewise, the research introduces the first NepaliGPT benchmark dataset comprised of 4,296 question-answer pairs in the Nepali language. The proposed LLM NepaliGPT achieves the following metrics in text generation: Perplexity of 26.32245, ROUGE-1 score of 0.2604, causal coherence of 81.25\%, and causal consistency of 85.41\%.