Exploring Data and Parameter Efficient Strategies for Arabic Dialect Identifications
This work addresses dialect identification for Arabic language processing, but it is incremental as it applies existing parameter-efficient techniques to this domain.
The paper explored data-efficient and parameter-efficient methods for Arabic Dialect Identification, finding that LLMs struggle with dialect nuances in few-shot or zero-shot setups, while LoRA-based fine-tuned models performed best, surpassing full fine-tuning.
This paper discusses our exploration of different data-efficient and parameter-efficient approaches to Arabic Dialect Identification (ADI). In particular, we investigate various soft-prompting strategies, including prefix-tuning, prompt-tuning, P-tuning, and P-tuning V2, as well as LoRA reparameterizations. For the data-efficient strategy, we analyze hard prompting with zero-shot and few-shot inferences to analyze the dialect identification capabilities of Large Language Models (LLMs). For the parameter-efficient PEFT approaches, we conducted our experiments using Arabic-specific encoder models on several major datasets. We also analyzed the n-shot inferences on open-source decoder-only models, a general multilingual model (Phi-3.5), and an Arabic-specific one(SILMA). We observed that the LLMs generally struggle to differentiate the dialectal nuances in the few-shot or zero-shot setups. The soft-prompted encoder variants perform better, while the LoRA-based fine-tuned models perform best, even surpassing full fine-tuning.