Combinatorial Topic Models using Small-Variance Asymptotics
This provides a new approach for unsupervised learning in text analysis, though it is incremental as it builds on LDA foundations.
The authors tackled topic modeling by reformulating it as a combinatorial optimization problem using small-variance asymptotics from LDA, resulting in a fast and high-quality algorithm that is competitive with existing LDA-based methods.
Topic models have emerged as fundamental tools in unsupervised machine learning. Most modern topic modeling algorithms take a probabilistic view and derive inference algorithms based on Latent Dirichlet Allocation (LDA) or its variants. In contrast, we study topic modeling as a combinatorial optimization problem, and propose a new objective function derived from LDA by passing to the small-variance limit. We minimize the derived objective by using ideas from combinatorial optimization, which results in a new, fast, and high-quality topic modeling algorithm. In particular, we show that our results are competitive with popular LDA-based topic modeling approaches, and also discuss the (dis)similarities between our approach and its probabilistic counterparts.