MACE: A Flexible Framework for Membership Privacy Estimation in Generative Models
This work addresses the need for formal privacy assessment in generative models used for sharing sensitive data, providing a flexible framework that improves upon rigid assumptions in existing methods, though it is incremental in advancing privacy estimation techniques.
The authors tackled the problem of estimating membership privacy risks in generative models, which lack formal privacy guarantees despite high sample quality, by proposing a flexible framework that formulates risk as a statistical divergence and offers sample-based estimations with theoretical guarantees, achieving more realistic assumptions compared to prior work.
Generative machine learning models are being increasingly viewed as a way to share sensitive data between institutions. While there has been work on developing differentially private generative modeling approaches, these approaches generally lead to sub-par sample quality, limiting their use in real world applications. Another line of work has focused on developing generative models which lead to higher quality samples but currently lack any formal privacy guarantees. In this work, we propose the first formal framework for membership privacy estimation in generative models. We formulate the membership privacy risk as a statistical divergence between training samples and hold-out samples, and propose sample-based methods to estimate this divergence. Compared to previous works, our framework makes more realistic and flexible assumptions. First, we offer a generalizable metric as an alternative to the accuracy metric especially for imbalanced datasets. Second, we loosen the assumption of having full access to the underlying distribution from previous studies , and propose sample-based estimations with theoretical guarantees. Third, along with the population-level membership privacy risk estimation via the optimal membership advantage, we offer the individual-level estimation via the individual privacy risk. Fourth, our framework allows adversaries to access the trained model via a customized query, while prior works require specific attributes.