A Large Language Model Guided Topic Refinement Mechanism for Short Text Modeling
This addresses the challenge of capturing semantic patterns in sparse short texts like tweets for applications in social trend analysis, though it is an incremental improvement as a post-processing method.
The paper tackles the problem of inaccurate topic modeling in short texts by introducing a model-agnostic Topic Refinement mechanism that uses Large Language Models to identify and replace intruder words in extracted topics, resulting in improved topic quality and performance in classification tasks across four datasets.
Modeling topics effectively in short texts, such as tweets and news snippets, is crucial to capturing rapidly evolving social trends. Existing topic models often struggle to accurately capture the underlying semantic patterns of short texts, primarily due to the sparse nature of such data. This nature of texts leads to an unavoidable lack of co-occurrence information, which hinders the coherence and granularity of mined topics. This paper introduces a novel model-agnostic mechanism, termed Topic Refinement, which leverages the advanced text comprehension capabilities of Large Language Models (LLMs) for short-text topic modeling. Unlike traditional methods, this post-processing mechanism enhances the quality of topics extracted by various topic modeling methods through prompt engineering. We guide LLMs in identifying semantically intruder words within the extracted topics and suggesting coherent alternatives to replace these words. This process mimics human-like identification, evaluation, and refinement of the extracted topics. Extensive experiments on four diverse datasets demonstrate that Topic Refinement boosts the topic quality and improves the performance in topic-related text classification tasks.