VinaLLaMA: LLaMA-based Vietnamese Foundation Model
This provides a versatile, open-weight resource for Vietnamese AI applications, addressing a domain-specific need.
The paper tackles the lack of advanced Vietnamese language models by developing VinaLLaMA, a LLaMA-2-based model trained on 800 billion tokens, which achieves state-of-the-art results on benchmarks like VLSP, VMLU, and Vicuna Benchmark Vietnamese.
In this technical report, we present VinaLLaMA, an open-weight, state-of-the-art (SOTA) Large Language Model for the Vietnamese language, built upon LLaMA-2 with an additional 800 billion trained tokens. VinaLLaMA not only demonstrates fluency in Vietnamese but also exhibits a profound understanding of Vietnamese culture, making it a truly indigenous model. VinaLLaMA-7B-chat, trained on 1 million high-quality synthetic samples, achieves SOTA results on key benchmarks, including VLSP, VMLU, and Vicuna Benchmark Vietnamese, marking a significant advancement in the Vietnamese AI landscape and offering a versatile resource for various applications.