Continuous Semantic Topic Embedding Model Using Variational Autoencoder
This work addresses the challenge of improving topic coherence and semantic interpretation in topic modeling for natural language processing applications, though it is incremental as it builds on existing variational autoencoder and embedding methods.
The paper tackles the problem of modeling latent topics in documents by proposing a continuous semantic topic embedding model (CSTEM) that uses a variational autoencoder with a semantic distance function, achieving performance comparable to state-of-the-art models on datasets like 20 Newsgroup and NIPS papers.
This paper proposes the continuous semantic topic embedding model (CSTEM) which finds latent topic variables in documents using continuous semantic distance function between the topics and the words by means of the variational autoencoder(VAE). The semantic distance could be represented by any symmetric bell-shaped geometric distance function on the Euclidean space, for which the Mahalanobis distance is used in this paper. In order for the semantic distance to perform more properly, we newly introduce an additional model parameter for each word to take out the global factor from this distance indicating how likely it occurs regardless of its topic. It certainly improves the problem that the Gaussian distribution which is used in previous topic model with continuous word embedding could not explain the semantic relation correctly and helps to obtain the higher topic coherence. Through the experiments with the dataset of 20 Newsgroup, NIPS papers and CNN/Dailymail corpus, the performance of the recent state-of-the-art models is accomplished by our model as well as generating topic embedding vectors which makes possible to observe where the topic vectors are embedded with the word vectors in the real Euclidean space and how the topics are related each other semantically.