Statistical Theory of Differentially Private Marginal-based Data Synthesis Algorithms
This work addresses the need for theoretical foundations in privacy-preserving data synthesis, which is crucial for data scientists and organizations handling sensitive high-dimensional data, though it is incremental as it builds on existing marginal-based methods.
The paper tackles the lack of statistical theory for differentially private marginal-based data synthesis algorithms, establishing rigorous accuracy guarantees for Bayesian network-based methods with error bounds measured by total variation or L² distance, and deriving an upper bound for utility error in downstream tasks along with a lower bound for TV accuracy for all ε-DP generators.
Marginal-based methods achieve promising performance in the synthetic data competition hosted by the National Institute of Standards and Technology (NIST). To deal with high-dimensional data, the distribution of synthetic data is represented by a probabilistic graphical model (e.g., a Bayesian network), while the raw data distribution is approximated by a collection of low-dimensional marginals. Differential privacy (DP) is guaranteed by introducing random noise to each low-dimensional marginal distribution. Despite its promising performance in practice, the statistical properties of marginal-based methods are rarely studied in the literature. In this paper, we study DP data synthesis algorithms based on Bayesian networks (BN) from a statistical perspective. We establish a rigorous accuracy guarantee for BN-based algorithms, where the errors are measured by the total variation (TV) distance or the $L^2$ distance. Related to downstream machine learning tasks, an upper bound for the utility error of the DP synthetic data is also derived. To complete the picture, we establish a lower bound for TV accuracy that holds for every $ε$-DP synthetic data generator.