Coherence-Aware Neural Topic Modeling
This work addresses the challenge of producing more semantically meaningful topics in topic modeling, which is incremental as it builds on existing neural variational inference frameworks.
The authors tackled the problem of improving topic coherence in neural topic models by incorporating a coherence objective into training, achieving similar perplexity but substantially higher coherence compared to baselines.
Topic models are evaluated based on their ability to describe documents well (i.e. low perplexity) and to produce topics that carry coherent semantic meaning. In topic modeling so far, perplexity is a direct optimization target. However, topic coherence, owing to its challenging computation, is not optimized for and is only evaluated after training. In this work, under a neural variational inference framework, we propose methods to incorporate a topic coherence objective into the training process. We demonstrate that such a coherence-aware topic model exhibits a similar level of perplexity as baseline models but achieves substantially higher topic coherence.