ClusterFusion: Hybrid Clustering with Embedding Guidance and LLM Adaptation
This addresses the challenge of domain-specific text clustering for NLP practitioners, offering a novel approach that leverages LLMs more directly, though it builds incrementally on prior work using LLMs as auxiliary modules.
The paper tackles the problem of text clustering in domain-specific contexts where traditional methods struggle without fine-tuning, proposing ClusterFusion, a hybrid framework that uses LLMs as the clustering core guided by embeddings, achieving state-of-the-art performance on standard benchmarks and substantial gains in specialized domains.
Text clustering is a fundamental task in natural language processing, yet traditional clustering algorithms with pre-trained embeddings often struggle in domain-specific contexts without costly fine-tuning. Large language models (LLMs) provide strong contextual reasoning, yet prior work mainly uses them as auxiliary modules to refine embeddings or adjust cluster boundaries. We propose ClusterFusion, a hybrid framework that instead treats the LLM as the clustering core, guided by lightweight embedding methods. The framework proceeds in three stages: embedding-guided subset partition, LLM-driven topic summarization, and LLM-based topic assignment. This design enables direct incorporation of domain knowledge and user preferences, fully leveraging the contextual adaptability of LLMs. Experiments on three public benchmarks and two new domain-specific datasets demonstrate that ClusterFusion not only achieves state-of-the-art performance on standard tasks but also delivers substantial gains in specialized domains. To support future work, we release our newly constructed dataset and results on all benchmarks.