DPAF: Image Synthesis via Differentially Private Aggregation in Forward Phase
This work addresses the challenge of generating useful private synthetic images for data release, representing an incremental improvement over existing methods.
The paper tackles the problem of low utility in differentially private synthetic image generation, particularly for high-resolution images, by proposing DPAF, which improves synthetic data quality through forward-phase aggregation and asymmetric training, achieving better performance on datasets up to 128x128 resolution.
Differentially private synthetic data is a promising alternative for sensitive data release. Many differentially private generative models have been proposed in the literature. Unfortunately, they all suffer from the low utility of the synthetic data, particularly for images of high resolutions. Here, we propose DPAF, an effective differentially private generative model for high-dimensional image synthesis. Different from the prior private stochastic gradient descent-based methods that add Gaussian noises in the backward phase during the model training, DPAF adds a differentially private feature aggregation in the forward phase, bringing advantages, including the reduction of information loss in gradient clipping and low sensitivity for the aggregation. Moreover, as an improper batch size has an adverse impact on the utility of synthetic data, DPAF also tackles the problem of setting a proper batch size by proposing a novel training strategy that asymmetrically trains different parts of the discriminator. We extensively evaluate different methods on multiple image datasets (up to images of 128x128 resolution) to demonstrate the performance of DPAF.