Open Llama2 Model for the Lithuanian Language
This provides a resource for Lithuanian language processing, but it is incremental as it adapts existing methods to a new language.
The authors developed the first open Llama2-based large language models for Lithuanian, along with a Q/A dataset and benchmark translations, and found that high-quality pretraining datasets are crucial for efficient performance on language understanding tasks.
In this paper, we propose and describe the first open Llama2 large language models (LLMs) for the Lithuanian language, including an accompanying question/answer (Q/A) dataset and translations of popular LLM benchmarks. We provide a brief review of open regional LLMs and detailed information on the proposed LLMs and their training process. We also conduct an empirical evaluation, comparing the perplexities of the proposed LLMs with those of other modern open LLMs. In addition, benchmarking the proposed LLMs against language understanding tasks reveals that high-quality pretraining datasets may be essential for achieving models that perform efficiently on these benchmarks. The full realisations of the described LLMs are available in the accompanying open repository~\url{https://huggingface.co/neurotechnology}.