Rethinking Collapsed Variational Bayes Inference for LDA
This work provides theoretical insights into CVB0 inference for LDA, which is incremental as it clarifies existing methods without introducing new applications or broad improvements.
The paper tackled the problem of interpreting collapsed variational Bayes inference for LDA by proposing a novel interpretation using a zero-order Taylor expansion approximation and analyzing it with alpha-divergence, showing that CVB0 inference involves two divergence projections at alpha=1 and -1.
We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two different divergence projections: alpha=1 and -1. This interpretation will help shed light on CVB0 works.