Multivariate Gaussian Topic Modelling: A novel approach to discover topics with greater semantic coherence
This work addresses the challenge of discovering semantically coherent topics in text mining, particularly for improving interpretability over existing models like LDA and LSA, though it appears incremental in its methodological approach.
The paper tackles the problem of topic interpretability and performance in short texts by proposing a Multivariate Gaussian Topic Model (MGTM), which achieves a mean topic coherence of 0.7 and median of 0.76, outperforming four benchmark models on the 20 newsgroups dataset.
An important aspect of text mining involves information retrieval in form of discovery of semantic themes (topics) from documents using topic modelling. While generative topic models like Latent Dirichlet Allocation (LDA) or Latent Semantic Analysis (LSA) elegantly model topics as probability distributions and are useful in identifying latent topics from large document corpora with minimal supervision, they suffer from difficulty in topic interpretability and reduced performance in shorter texts. Here we propose a novel Multivariate Gaussian Topic Model (MGTM). In this approach topics are presented as Multivariate Gaussian Distributions and documents as Gaussian Mixture Models. Applying EM algorithm on a document corpus, the various constituent Multivariate Gaussian distributions corresponding to the latent topics and their respective parameters are identified. Analysis of the parameters of each distribution helps identify the respective topic keywords, and from these key-words topic annotations are carried out. This approach is applied on 20 newsgroups dataset to demonstrate the interpretability benefits vis-`a-vis 4 other benchmark models. The effectiveness of this model in capturing the semantic theme of the topics with high interpretability is examined by calculating the topic coherence and comparing the coherence values with benchmark models. This model achieves a highest mean topic coherence (0.7) and median topic coherence (0.76) vis-`a-vis the benchmark models, demonstrating high effectiveness in identifying interpretable, semantically coherent topics.