Novi jezički modeli za srpski jezik
This work addresses the need for improved language models for the Serbian language, which is incremental as it builds on existing transformer-based approaches.
The paper tackles the development and evaluation of transformer-based language models for the Serbian language, presenting new models for text generation and vectorization and comparing ten vectorization models on four NLP tasks to analyze performance based on model size and training data.
The paper will briefly present the development history of transformer-based language models for the Serbian language. Several new models for text generation and vectorization, trained on the resources of the Society for Language Resources and Technologies, will also be presented. Ten selected vectorization models for Serbian, including two new ones, will be compared on four natural language processing tasks. Paper will analyze which models are the best for each selected task, how does their size and the size of their training sets affect the performance on those tasks, and what is the optimal setting to train the best language models for the Serbian language.