SMTCE: A Social Media Text Classification Evaluation Benchmark and BERTology Models for Vietnamese
This provides a standardized evaluation framework for Vietnamese, a low-resource language, benefiting researchers in NLP and computational linguistics, though it is incremental as it adapts existing methods to a new domain.
The authors tackled the lack of benchmarks for social media text classification in Vietnamese by introducing the SMTCE benchmark, and found that monolingual BERT-based models outperform multilingual ones, achieving state-of-the-art results across all tasks.
Text classification is a typical natural language processing or computational linguistics task with various interesting applications. As the number of users on social media platforms increases, data acceleration promotes emerging studies on Social Media Text Classification (SMTC) or social media text mining on these valuable resources. In contrast to English, Vietnamese, one of the low-resource languages, is still not concentrated on and exploited thoroughly. Inspired by the success of the GLUE, we introduce the Social Media Text Classification Evaluation (SMTCE) benchmark, as a collection of datasets and models across a diverse set of SMTC tasks. With the proposed benchmark, we implement and analyze the effectiveness of a variety of multilingual BERT-based models (mBERT, XLM-R, and DistilmBERT) and monolingual BERT-based models (PhoBERT, viBERT, vELECTRA, and viBERT4news) for tasks in the SMTCE benchmark. Monolingual models outperform multilingual models and achieve state-of-the-art results on all text classification tasks. It provides an objective assessment of multilingual and monolingual BERT-based models on the benchmark, which will benefit future studies about BERTology in the Vietnamese language.