A new LDA formulation with covariates
This work addresses the need for interpretable covariate integration in LDA for researchers analyzing discrete data across domains like text, retail, and ecology, though it is incremental as it builds on existing LDA frameworks.
The authors tackled the problem of incorporating covariates into Latent Dirichlet Allocation (LDA) for mixed-membership clustering by proposing a new formulation that embeds negative binomial regression, enabling interpretation of coefficients and analysis of cluster-specific abundances. They demonstrated the model's effectiveness through simulations retrieving true parameter values and applied it to real datasets in text-mining, grocery shopping, and ecology.
The Latent Dirichlet Allocation (LDA) model is a popular method for creating mixed-membership clusters. Despite having been originally developed for text analysis, LDA has been used for a wide range of other applications. We propose a new formulation for the LDA model which incorporates covariates. In this model, a negative binomial regression is embedded within LDA, enabling straight-forward interpretation of the regression coefficients and the analysis of the quantity of cluster-specific elements in each sampling units (instead of the analysis being focused on modeling the proportion of each cluster, as in Structural Topic Models). We use slice sampling within a Gibbs sampling algorithm to estimate model parameters. We rely on simulations to show how our algorithm is able to successfully retrieve the true parameter values and the ability to make predictions for the abundance matrix using the information given by the covariates. The model is illustrated using real data sets from three different areas: text-mining of Coronavirus articles, analysis of grocery shopping baskets, and ecology of tree species on Barro Colorado Island (Panama). This model allows the identification of mixed-membership clusters in discrete data and provides inference on the relationship between covariates and the abundance of these clusters.