Sabiá-2: A New Generation of Portuguese Large Language Models
This addresses the need for accessible and high-performing Portuguese language models for educational and professional applications, though it is incremental as it adapts existing methods to a new language.
The researchers tackled the problem of developing large language models for Portuguese by introducing Sabiá-2, which matches or surpasses GPT-4 in 23 out of 64 exams and outperforms GPT-3.5 in 58 out of 64 exams, while offering a 10 times cheaper price per token than GPT-4.
We introduce Sabiá-2, a family of large language models trained on Portuguese texts. The models are evaluated on a diverse range of exams, including entry-level tests for Brazilian universities, professional certification exams, and graduate-level exams for various disciplines such as accounting, economics, engineering, law and medicine. Our results reveal that our best model so far, Sabiá-2 Medium, matches or surpasses GPT-4's performance in 23 out of 64 exams and outperforms GPT-3.5 in 58 out of 64 exams. Notably, specialization has a significant impact on a model's performance without the need to increase its size, allowing us to offer Sabiá-2 Medium at a price per token that is 10 times cheaper than GPT-4. Finally, we identified that math and coding are key abilities that need improvement.