Holistic Evaluations of Topic Models
This work addresses the need for better scrutiny and understanding of topic models for researchers and general users, but it is incremental as it builds on existing evaluation methods.
The paper tackles the problem of topic models being used as black boxes by evaluating them from a database perspective, analyzing 1140 BERTopic model runs to identify trade-offs in parameter optimization and implications for interpretation and responsible use.
Topic models are gaining increasing commercial and academic interest for their ability to summarize large volumes of unstructured text. As unsupervised machine learning methods, they enable researchers to explore data and help general users understand key themes in large text collections. However, they risk becoming a 'black box', where users input data and accept the output as an accurate summary without scrutiny. This article evaluates topic models from a database perspective, drawing insights from 1140 BERTopic model runs. The goal is to identify trade-offs in optimizing model parameters and to reflect on what these findings mean for the interpretation and responsible use of topic models