Prediction Focused Topic Models via Feature Selection
This work addresses the challenge for researchers and practitioners in natural language processing who need interpretable topic models that also perform well in prediction tasks, though it appears incremental as it builds on existing supervised topic modeling methods.
The authors tackled the problem of balancing prediction quality and interpretability in supervised topic models by introducing a prediction-focused topic model that uses feature selection to retain only vocabulary terms that improve or do not hinder prediction performance. They demonstrated that this approach learns more coherent topics while maintaining competitive predictions on several datasets.
Supervised topic models are often sought to balance prediction quality and interpretability. However, when models are (inevitably) misspecified, standard approaches rarely deliver on both. We introduce a novel approach, the prediction-focused topic model, that uses the supervisory signal to retain only vocabulary terms that improve, or at least do not hinder, prediction performance. By removing terms with irrelevant signal, the topic model is able to learn task-relevant, coherent topics. We demonstrate on several data sets that compared to existing approaches, prediction-focused topic models learn much more coherent topics while maintaining competitive predictions.