Managing sparsity, time, and quality of inference in topic models
This work addresses the problem of improving inference efficiency and sparsity in topic models for researchers and practitioners in machine learning and natural language processing, representing an incremental advancement.
The authors tackled the challenge of efficient inference in probabilistic topic models, particularly for achieving sparse latent document representations, by introducing the FW framework, which demonstrated linear convergence and allowed direct trade-offs between sparsity, quality, and time.
Inference is an integral part of probabilistic topic models, but is often non-trivial to derive an efficient algorithm for a specific model. It is even much more challenging when we want to find a fast inference algorithm which always yields sparse latent representations of documents. In this article, we introduce a simple framework for inference in probabilistic topic models, denoted by FW. This framework is general and flexible enough to be easily adapted to mixture models. It has a linear convergence rate, offers an easy way to incorporate prior knowledge, and provides us an easy way to directly trade off sparsity against quality and time. We demonstrate the goodness and flexibility of FW over existing inference methods by a number of tasks. Finally, we show how inference in topic models with nonconjugate priors can be done efficiently.