Model Selection for Topic Models via Spectral Decomposition
This provides a theoretical foundation for model selection in topic models, which is incremental but addresses a key practical challenge in text analysis.
The paper tackles the problem of selecting the number of topics in latent Dirichlet allocation by deriving theoretical upper and lower bounds based on spectral decomposition, with experiments showing these bounds are accurate and tight.
Topic models have achieved significant successes in analyzing large-scale text corpus. In practical applications, we are always confronted with the challenge of model selection, i.e., how to appropriately set the number of topics. Following recent advances in topic model inference via tensor decomposition, we make a first attempt to provide theoretical analysis on model selection in latent Dirichlet allocation. Under mild conditions, we derive the upper bound and lower bound on the number of topics given a text collection of finite size. Experimental results demonstrate that our bounds are accurate and tight. Furthermore, using Gaussian mixture model as an example, we show that our methodology can be easily generalized to model selection analysis for other latent models.