Boosting Short Text Classification with Multi-Source Information Exploration and Dual-Level Contrastive Learning
This work addresses short text classification, a challenging NLP problem due to sparse semantics and insufficient labels, with incremental improvements through a novel hybrid method.
The authors tackled short text classification by proposing MI-DELIGHT, which uses multi-source information exploration and dual-level contrastive learning to address semantic sparsity and limited labeled data, achieving significant performance improvements over previous models and even outperforming large language models on some datasets.
Short text classification, as a research subtopic in natural language processing, is more challenging due to its semantic sparsity and insufficient labeled samples in practical scenarios. We propose a novel model named MI-DELIGHT for short text classification in this work. Specifically, it first performs multi-source information (i.e., statistical information, linguistic information, and factual information) exploration to alleviate the sparsity issues. Then, the graph learning approach is adopted to learn the representation of short texts, which are presented in graph forms. Moreover, we introduce a dual-level (i.e., instance-level and cluster-level) contrastive learning auxiliary task to effectively capture different-grained contrastive information within massive unlabeled data. Meanwhile, previous models merely perform the main task and auxiliary tasks in parallel, without considering the relationship among tasks. Therefore, we introduce a hierarchical architecture to explicitly model the correlations between tasks. We conduct extensive experiments across various benchmark datasets, demonstrating that MI-DELIGHT significantly surpasses previous competitive models. It even outperforms popular large language models on several datasets.