Camoscio: an Italian Instruction-tuned LLaMA
This work addresses the problem of limited accessible resources for Italian language processing, though it is incremental as it adapts existing methods to a new language.
The authors tackled the lack of open, Italian-specific large language models by introducing Camoscio, an instruction-tuned LLaMA variant, which achieves competitive zero-shot performance on Italian downstream tasks compared to task-specific models.
In recent years Large Language Models (LLMs) have increased the state of the art on several natural language processing tasks. However, their accessibility is often limited to paid API services, posing challenges for researchers in conducting extensive investigations. On the other hand, while some open-source models have been proposed by the community, they are typically English-centric or multilingual without a specific adaptation for the Italian language. In an effort to democratize the available and open resources for the Italian language, in this paper we introduce Camoscio: a language model specifically tuned to follow users' prompts in Italian. Specifically, we finetuned the smallest variant of LLaMA (7b) with LoRA on a corpus of instruction prompts translated to Italian via ChatGPT. Results indicate that the model's zero-shot performance on various downstream tasks in Italian competes favorably with existing models specifically finetuned for those tasks. All the artifacts (code, dataset, model) are released to the community at the following url: https://github.com/teelinsan/camoscio