Topic Analysis for Text with Side Data
This addresses problems in text analysis and recommendation systems, but it is incremental as it builds on existing topic modeling and neural network approaches.
The paper tackled limitations of latent factor models like cold-start and non-transparency by introducing a hybrid generative probabilistic model that combines a neural network with a latent topic model, showing it outperforms standard LDA and DMR in topic grouping, model perplexity, classification, and comment generation on several datasets.
Although latent factor models (e.g., matrix factorization) obtain good performance in predictions, they suffer from several problems including cold-start, non-transparency, and suboptimal recommendations. In this paper, we employ text with side data to tackle these limitations. We introduce a hybrid generative probabilistic model that combines a neural network with a latent topic model, which is a four-level hierarchical Bayesian model. In the model, each document is modeled as a finite mixture over an underlying set of topics and each topic is modeled as an infinite mixture over an underlying set of topic probabilities. Furthermore, each topic probability is modeled as a finite mixture over side data. In the context of text, the neural network provides an overview distribution about side data for the corresponding text, which is the prior distribution in LDA to help perform topic grouping. The approach is evaluated on several different datasets, where the model is shown to outperform standard LDA and Dirichlet-multinomial regression (DMR) in terms of topic grouping, model perplexity, classification and comment generation.