PersianMind: A Cross-Lingual Persian-English Large Language Model
This addresses the need for high-quality open-source language models in Persian, offering a domain-specific improvement for Persian-speaking users and applications.
The authors tackled the problem of poor performance of open-source large language models in non-English languages by introducing PersianMind, a bilingual Persian-English model that achieves comparable performance to GPT-3.5-turbo in Persian while preserving English knowledge.
Large language models demonstrate remarkable proficiency in various linguistic tasks and have extensive knowledge across various domains. Although they perform best in English, their ability in other languages is notable too. In contrast, open-source models, such as LLaMa, are primarily trained on English datasets, resulting in poor performance in non-English languages. In this paper, we introduce PersianMind, an open-source bilingual large language model which demonstrates comparable performance to closed-source GPT-3.5-turbo in the Persian language. By expanding LLaMa2's vocabulary with 10,000 Persian tokens and training it on a dataset comprising nearly 2 billion Persian tokens, we show that our approach preserves the model's English knowledge and employs transfer learning to excel at transferring task knowledge from one language to another.