Membership Inference Attacks and Privacy in Topic Modeling
This work addresses privacy risks for users of topic modeling in text analysis, showing that vulnerabilities extend beyond large neural models, though it is incremental in applying known attack and mitigation techniques to a new model type.
The paper tackles the problem of privacy vulnerabilities in simpler generative models like topic models, specifically Latent Dirichlet Allocation, by proposing a membership inference attack that can confidently identify training data members, and it explores differentially private topic modeling with a framework that improves privacy while maintaining utility.
Recent research shows that large language models are susceptible to privacy attacks that infer aspects of the training data. However, it is unclear if simpler generative models, like topic models, share similar vulnerabilities. In this work, we propose an attack against topic models that can confidently identify members of the training data in Latent Dirichlet Allocation. Our results suggest that the privacy risks associated with generative modeling are not restricted to large neural models. Additionally, to mitigate these vulnerabilities, we explore differentially private (DP) topic modeling. We propose a framework for private topic modeling that incorporates DP vocabulary selection as a pre-processing step, and show that it improves privacy while having limited effects on practical utility.