ANTM: An Aligned Neural Topic Model for Exploring Evolving Topics
This work addresses the need for scalable and interpretable dynamic topic modeling for researchers analyzing large-scale textual data, representing an incremental improvement over existing methods.
The paper tackles the problem of modeling evolving topics in textual data by introducing ANTM, a dynamic topic model that uses LLMs and an overlapping sliding window algorithm to maintain temporal continuity and align topics across time periods, outperforming probabilistic models in coherence and diversity metrics on four datasets.
This paper presents an algorithmic family of dynamic topic models called Aligned Neural Topic Models (ANTM), which combine novel data mining algorithms to provide a modular framework for discovering evolving topics. ANTM maintains the temporal continuity of evolving topics by extracting time-aware features from documents using advanced pre-trained Large Language Models (LLMs) and employing an overlapping sliding window algorithm for sequential document clustering. This overlapping sliding window algorithm identifies a different number of topics within each time frame and aligns semantically similar document clusters across time periods. This process captures emerging and fading trends across different periods and allows for a more interpretable representation of evolving topics. Experiments on four distinct datasets show that ANTM outperforms probabilistic dynamic topic models in terms of topic coherence and diversity metrics. Moreover, it improves the scalability and flexibility of dynamic topic models by being accessible and adaptable to different types of algorithms. Additionally, a Python package is developed for researchers and scientists who wish to study the trends and evolving patterns of topics in large-scale textual data.