Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks
This work addresses the challenge of developing comparable multimodal models for Arabic, a language with a large speaker population, by providing models and a benchmark, though it is incremental as it adapts existing MLLM approaches to a new language.
The authors tackled the lack of high-quality multimodal resources for non-English languages by introducing Peacock, a family of Arabic multimodal large language models, which demonstrated solid performance on visual reasoning tasks and emerging dialectal potential, along with a new benchmark called Henna for culturally-aware Arabic MLLMs.
Multimodal large language models (MLLMs) have proven effective in a wide range of tasks requiring complex reasoning and linguistic comprehension. However, due to a lack of high-quality multimodal resources in languages other than English, success of MLLMs remains relatively limited to English-based settings. This poses significant challenges in developing comparable models for other languages, including even those with large speaker populations such as Arabic. To alleviate this challenge, we introduce a comprehensive family of Arabic MLLMs, dubbed \textit{Peacock}, with strong vision and language capabilities. Through comprehensive qualitative and quantitative analysis, we demonstrate the solid performance of our models on various visual reasoning tasks and further show their emerging dialectal potential. Additionally, we introduce ~\textit{Henna}, a new benchmark specifically designed for assessing MLLMs on aspects related to Arabic culture, setting the first stone for culturally-aware Arabic MLLMs.The GitHub repository for the \textit{Peacock} project is available at \url{https://github.com/UBC-NLP/peacock}.