Evaluating Dynamic Topic Models
This work addresses a gap in evaluating dynamic topic models for researchers in natural language processing and text mining, though it is incremental as it builds on existing models.
The authors tackled the problem of lacking quantitative measures to evaluate topic progression in dynamic topic models by proposing a novel evaluation measure that analyzes topic quality changes over time and combines it with temporal consistency, demonstrating its utility on synthetic and existing data and showing good correlation with human judgment.
There is a lack of quantitative measures to evaluate the progression of topics through time in dynamic topic models (DTMs). Filling this gap, we propose a novel evaluation measure for DTMs that analyzes the changes in the quality of each topic over time. Additionally, we propose an extension combining topic quality with the model's temporal consistency. We demonstrate the utility of the proposed measure by applying it to synthetic data and data from existing DTMs. We also conducted a human evaluation, which indicates that the proposed measure correlates well with human judgment. Our findings may help in identifying changing topics, evaluating different DTMs, and guiding future research in this area.