RigoBERTa: A State-of-the-Art Language Model For Spanish
This provides an incremental improvement for Spanish natural language processing by enhancing performance on specific tasks.
The paper tackled the problem of developing a state-of-the-art language model for Spanish by introducing RigoBERTa, which outperformed existing models like MarIA, BERTIN, and BETO in 10 out of 13 NLU tasks, achieving new SOTA results.
This paper presents RigoBERTa, a State-of-the-Art Language Model for Spanish. RigoBERTa is trained over a well-curated corpus formed up from different subcorpora with key features. It follows the DeBERTa architecture, which has several advantages over other architectures of similar size as BERT or RoBERTa. RigoBERTa performance is assessed over 13 NLU tasks in comparison with other available Spanish language models, namely, MarIA, BERTIN and BETO. RigoBERTa outperformed the three models in 10 out of the 13 tasks, achieving new "State-of-the-Art" results.