Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics
This work addresses the need for simpler and more effective topic modeling methods for researchers and practitioners in natural language processing, though it is incremental as it builds on existing embedding techniques.
The paper tackles the problem of generating coherent and interpretable topics by showing that directly clustering high-quality sentence embeddings with a word selection method outperforms Neural Topic Models, achieving higher topic coherence and diversity with improved efficiency and simplicity.
Recent work incorporates pre-trained word embeddings such as BERT embeddings into Neural Topic Models (NTMs), generating highly coherent topics. However, with high-quality contextualized document representations, do we really need sophisticated neural models to obtain coherent and interpretable topics? In this paper, we conduct thorough experiments showing that directly clustering high-quality sentence embeddings with an appropriate word selecting method can generate more coherent and diverse topics than NTMs, achieving also higher efficiency and simplicity.