Practical Collapsed Stochastic Variational Inference for the HDP
This work addresses a gap in scalable Bayesian inference for topic modeling, though it appears incremental as it extends existing methods to a collapsed setting.
The paper tackles the problem of applying collapsed stochastic variational inference to the hierarchical Dirichlet process (HDP) for non-parametric topic modeling, resulting in an online algorithm that shows promising improvement in predictive performance.
Recent advances have made it feasible to apply the stochastic variational paradigm to a collapsed representation of latent Dirichlet allocation (LDA). While the stochastic variational paradigm has successfully been applied to an uncollapsed representation of the hierarchical Dirichlet process (HDP), no attempts to apply this type of inference in a collapsed setting of non-parametric topic modeling have been put forward so far. In this paper we explore such a collapsed stochastic variational Bayes inference for the HDP. The proposed online algorithm is easy to implement and accounts for the inference of hyper-parameters. First experiments show a promising improvement in predictive performance.