Multi-environment Topic Models
This work addresses the need for interpretable topic modeling in multi-environment text data, such as political content, with incremental improvements over existing methods.
The paper tackles the problem of learning global and environment-specific topic representations from text with covariates, introducing the Multi-environment Topic Model (MTM) which outperforms baselines in and out-of-distribution and enables accurate causal effect discovery.
Probabilistic topic models are a powerful tool for extracting latent themes from large text datasets. In many text datasets, we also observe per-document covariates (e.g., source, style, political affiliation) that act as environments that modulate a "global" (environment-agnostic) topic representation. Accurately learning these representations is important for prediction on new documents in unseen environments and for estimating the causal effect of topics on real-world outcomes. To this end, we introduce the Multi-environment Topic Model (MTM), an unsupervised probabilistic model that separates global and environment-specific terms. Through experimentation on various political content, from ads to tweets and speeches, we show that the MTM produces interpretable global topics with distinct environment-specific words. On multi-environment data, the MTM outperforms strong baselines in and out-of-distribution. It also enables the discovery of accurate causal effects.