Private Synthetic Data for Multitask Learning and Marginal Queries
This work addresses the need for scalable and accurate private synthetic data generation for researchers and practitioners dealing with mixed-type datasets, though it is incremental as it builds on prior approaches by eliminating binning.
The authors tackled the problem of generating differentially private synthetic data that is simultaneously useful for multiple tasks, such as marginal queries and multitask machine learning, by developing an algorithm that directly handles numerical features without binning. The result is a method that runs 2-5x faster than comparable techniques and provides significant accuracy improvements in both marginal queries and linear prediction tasks for mixed-type datasets.
We provide a differentially private algorithm for producing synthetic data simultaneously useful for multiple tasks: marginal queries and multitask machine learning (ML). A key innovation in our algorithm is the ability to directly handle numerical features, in contrast to a number of related prior approaches which require numerical features to be first converted into {high cardinality} categorical features via {a binning strategy}. Higher binning granularity is required for better accuracy, but this negatively impacts scalability. Eliminating the need for binning allows us to produce synthetic data preserving large numbers of statistical queries such as marginals on numerical features, and class conditional linear threshold queries. Preserving the latter means that the fraction of points of each class label above a particular half-space is roughly the same in both the real and synthetic data. This is the property that is needed to train a linear classifier in a multitask setting. Our algorithm also allows us to produce high quality synthetic data for mixed marginal queries, that combine both categorical and numerical features. Our method consistently runs 2-5x faster than the best comparable techniques, and provides significant accuracy improvements in both marginal queries and linear prediction tasks for mixed-type datasets.