STF: Sentence Transformer Fine-Tuning For Topic Categorization With Limited Data
This addresses the problem of low performance in tweet topic classification due to data scarcity for researchers and practitioners, though it is incremental as it applies existing fine-tuning techniques to a specific domain.
The authors tackled topic classification from tweets with limited labeled data by proposing STF, a system that fine-tunes pretrained Sentence Transformers, achieving state-of-the-art performance on benchmark datasets without requiring large amounts of labeled data.
Nowadays, topic classification from tweets attracts considerable research attention. Different classification systems have been suggested thanks to these research efforts. Nevertheless, they face major challenges owing to low performance metrics due to the limited amount of labeled data. We propose Sentence Transformers Fine-tuning (STF), a topic detection system that leverages pretrained Sentence Transformers models and fine-tuning to classify topics from tweets accurately. Moreover, extensive parameter sensitivity analyses were conducted to finetune STF parameters for our topic classification task to achieve the best performance results. Experiments on two benchmark datasets demonstrated that (1) the proposed STF can be effectively used for classifying tweet topics and outperforms the latest state-of-the-art approaches, and (2) the proposed STF does not require a huge amount of labeled tweets to achieve good accuracy, which is a limitation of many state-of-the-art approaches. Our main contribution is the achievement of promising results in tweet topic classification by applying pretrained sentence transformers language models.