Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages
This work addresses the performance gap in LLMs for low-resource languages, specifically Amharic spoken by over 50 million people, but it is incremental as it applies existing methods to a new language and modality.
The authors tackled the problem of large language models (LLMs) struggling with low-resource languages like Amharic, which has limited training data, by training LLaMA-2 on Amharic using data augmentation and extending it to a multimodal model with image understanding, resulting in open-sourced models and a benchmarking dataset.
Large Language Models (LLMs) like GPT-4 and LLaMA have shown incredible proficiency at natural language processing tasks and have even begun to excel at tasks across other modalities such as vision and audio. Despite their success, LLMs often struggle to perform well on low-resource languages because there is so little training data available. This shortcoming is especially prevalent with open source models. In this work, we explore training LLaMA-2 to speak Amharic, a language which is spoken by over 50 million people world wide, but has orders of magnitude less data available than languages like English. We employ methods previously used for training LLMs on other languages with data scarcity, and use open source translation models to perform data augmentation and grow our dataset from millions of tokens to billions. We further enhance the capabilities of our model by connecting an image encoder and training on a translated visual instruction tuning dataset in the same manner as LLaVA, resulting in a multimodal Amharic LLM that can understand images along with text. We introduce an Amharic version of a popular benchmarking dataset to evaluate our work. Our models and dataset are open sourced and available on GitHub.