Efficient Correlated Topic Modeling with Topic Embedding
This work addresses a bottleneck in topic modeling for researchers and practitioners dealing with large-scale text data, offering a scalable solution with significant efficiency gains.
The paper tackles the computational inefficiency and poor scaling of correlated topic modeling by introducing a model that uses compact topic embeddings to capture correlations, achieving linear time complexity with respect to topic size. Experiments show it handles model and data scales several orders of magnitude larger than existing methods while maintaining competitive or superior performance in document classification and retrieval.
Correlated topic modeling has been limited to small model and problem sizes due to their high computational cost and poor scaling. In this paper, we propose a new model which learns compact topic embeddings and captures topic correlations through the closeness between the topic vectors. Our method enables efficient inference in the low-dimensional embedding space, reducing previous cubic or quadratic time complexity to linear w.r.t the topic size. We further speedup variational inference with a fast sampler to exploit sparsity of topic occurrence. Extensive experiments show that our approach is capable of handling model and data scales which are several orders of magnitude larger than existing correlation results, without sacrificing modeling quality by providing competitive or superior performance in document classification and retrieval.