Text Classification in the LLM Era -- Where do we stand?
This work provides guidance for practitioners developing text classification systems across languages, though it is incremental as it compares existing methods on new data.
The paper investigated the performance of large language models (LLMs) in text classification compared to other methods, finding that zero-shot approaches excel in sentiment classification but are outperformed by other methods for other tasks, with synthetic data from multiple LLMs yielding better classifiers than zero-shot open LLMs, and noting wide performance disparities across 8 languages.
Large Language Models revolutionized NLP and showed dramatic performance improvements across several tasks. In this paper, we investigated the role of such language models in text classification and how they compare with other approaches relying on smaller pre-trained language models. Considering 32 datasets spanning 8 languages, we compared zero-shot classification, few-shot fine-tuning and synthetic data based classifiers with classifiers built using the complete human labeled dataset. Our results show that zero-shot approaches do well for sentiment classification, but are outperformed by other approaches for the rest of the tasks, and synthetic data sourced from multiple LLMs can build better classifiers than zero-shot open LLMs. We also see wide performance disparities across languages in all the classification scenarios. We expect that these findings would guide practitioners working on developing text classification systems across languages.