Advancing Generative AI for Portuguese with Open Decoder Gervásio PT*
This work addresses the need for improved language technology for Portuguese speakers, though it is incremental as it builds on existing models and focuses on a specific language domain.
The paper tackles the problem of advancing generative AI for Portuguese by introducing Gervásio PT*, an open-source, instruction-tuned Transformer decoder model based on LLaMA 2 7B, which sets a new state of the art for Portuguese neural decoding and includes new instruction datasets.
To advance the neural decoding of Portuguese, in this paper we present a fully open Transformer-based, instruction-tuned decoder model that sets a new state of the art in this respect. To develop this decoder, which we named Gervásio PT*, a strong LLaMA~2 7B model was used as a starting point, and its further improvement through additional training was done over language resources that include new instruction data sets of Portuguese prepared for this purpose, which are also contributed in this paper. All versions of Gervásio are open source and distributed for free under an open license, including for either research or commercial usage, and can be run on consumer-grade hardware, thus seeking to contribute to the advancement of research and innovation in language technology for Portuguese.