Reinforcement Learning for Topic Models
This work addresses topic modeling for natural language processing, offering an incremental improvement by integrating reinforcement learning into an existing framework.
The authors tackled the problem of topic modeling by replacing the variational autoencoder in ProdLDA with a reinforcement learning policy, achieving results where their unsupervised model outperformed all other unsupervised models and performed on par with or better than most supervised models across 11 datasets.
We apply reinforcement learning techniques to topic modeling by replacing the variational autoencoder in ProdLDA with a continuous action space reinforcement learning policy. We train the system with a policy gradient algorithm REINFORCE. Additionally, we introduced several modifications: modernize the neural network architecture, weight the ELBO loss, use contextual embeddings, and monitor the learning process via computing topic diversity and coherence for each training step. Experiments are performed on 11 data sets. Our unsupervised model outperforms all other unsupervised models and performs on par with or better than most models using supervised labeling. Our model is outperformed on certain data sets by a model using supervised labeling and contrastive learning. We have also conducted an ablation study to provide empirical evidence of performance improvements from changes we made to ProdLDA and found that the reinforcement learning formulation boosts performance.