MetaLDA: a Topic Model that Efficiently Incorporates Meta information
This work addresses the challenge of enhancing topic modeling for documents with insufficient word-occurrence data by efficiently leveraging meta information, representing an incremental improvement over existing methods.
The authors tackled the problem of incorporating meta information into topic models to improve accuracy and topic quality, especially with sparse texts, and demonstrated that MetaLDA achieves comparable or improved performance in perplexity and topic quality while running significantly faster than other models.
Besides the text content, documents and their associated words usually come with rich sets of meta informa- tion, such as categories of documents and semantic/syntactic features of words, like those encoded in word embeddings. Incorporating such meta information directly into the generative process of topic models can improve modelling accuracy and topic quality, especially in the case where the word-occurrence information in the training data is insufficient. In this paper, we present a topic model, called MetaLDA, which is able to leverage either document or word meta information, or both of them jointly. With two data argumentation techniques, we can derive an efficient Gibbs sampling algorithm, which benefits from the fully local conjugacy of the model. Moreover, the algorithm is favoured by the sparsity of the meta information. Extensive experiments on several real world datasets demonstrate that our model achieves comparable or improved performance in terms of both perplexity and topic quality, particularly in handling sparse texts. In addition, compared with other models using meta information, our model runs significantly faster.