Developing and Evaluating Tiny to Medium-Sized Turkish BERT Models
This work provides smaller, efficient language models for Turkish, an incremental improvement for NLP applications in less-resourced languages.
The study developed and evaluated tiny to medium-sized Turkish BERT models to address the gap in less-resourced languages, achieving robust performance on tasks like sentiment analysis and zero-shot classification while maintaining computational efficiency.
This study introduces and evaluates tiny, mini, small, and medium-sized uncased Turkish BERT models, aiming to bridge the research gap in less-resourced languages. We trained these models on a diverse dataset encompassing over 75GB of text from multiple sources and tested them on several tasks, including mask prediction, sentiment analysis, news classification, and, zero-shot classification. Despite their smaller size, our models exhibited robust performance, including zero-shot task, while ensuring computational efficiency and faster execution times. Our findings provide valuable insights into the development and application of smaller language models, especially in the context of the Turkish language.