Context Reinforced Neural Topic Modeling over Short Texts
This work addresses a domain-specific problem for researchers and practitioners in natural language processing dealing with short text analysis, but it is incremental as it builds on existing neural topic modeling approaches.
The authors tackled the problem of feature sparsity in neural topic modeling for short texts by proposing a Context Reinforced Neural Topic Model (CRNTM), which uses pre-trained word embeddings and narrow topic inference, and validated its effectiveness on benchmark datasets for topic discovery and text classification.
As one of the prevalent topic mining tools, neural topic modeling has attracted a lot of interests for the advantages of high efficiency in training and strong generalisation abilities. However, due to the lack of context in each short text, the existing neural topic models may suffer from feature sparsity on such documents. To alleviate this issue, we propose a Context Reinforced Neural Topic Model (CRNTM), whose characteristics can be summarized as follows. Firstly, by assuming that each short text covers only a few salient topics, CRNTM infers the topic for each word in a narrow range. Secondly, our model exploits pre-trained word embeddings by treating topics as multivariate Gaussian distributions or Gaussian mixture distributions in the embedding space. Extensive experiments on two benchmark datasets validate the effectiveness of the proposed model on both topic discovery and text classification.