Continual Neural Topic Model
This addresses the challenge of maintaining long-term memory in topic modeling for applications requiring continuous data updates, representing an incremental improvement over existing methods.
The paper tackled the problem of learning new topic models without forgetting previously learned topics in continual learning, proposing the Continual Neural Topic Model (CoNTM) that outperformed dynamic topic models in topic quality and predictive perplexity while capturing topic changes online.
In continual learning, our aim is to learn a new task without forgetting what was learned previously. In topic models, this translates to learning new topic models without forgetting previously learned topics. Previous work either considered Dynamic Topic Models (DTMs), which learn the evolution of topics based on the entire training corpus at once, or Online Topic Models, which are updated continuously based on new data but do not have long-term memory. To fill this gap, we propose the Continual Neural Topic Model (CoNTM), which continuously learns topic models at subsequent time steps without forgetting what was previously learned. This is achieved using a global prior distribution that is continuously updated. In our experiments, CoNTM consistently outperformed the dynamic topic model in terms of topic quality and predictive perplexity while being able to capture topic changes online. The analysis reveals that CoNTM can learn more diverse topics and better capture temporal changes than existing methods.