Private GANs, Revisited
This work addresses the challenge of generating high-quality synthetic data with privacy guarantees for applications like healthcare or finance, though it is incremental as it builds on existing DPSGD methods.
The authors tackled the problem of training differentially private GANs by modifying the canonical approach, showing that adjusting discriminator update frequency and other training modifications significantly improves generation quality, outperforming alternative privatization schemes on standard benchmarks.
We show that the canonical approach for training differentially private GANs -- updating the discriminator with differentially private stochastic gradient descent (DPSGD) -- can yield significantly improved results after modifications to training. Specifically, we propose that existing instantiations of this approach neglect to consider how adding noise only to discriminator updates inhibits discriminator training, disrupting the balance between the generator and discriminator necessary for successful GAN training. We show that a simple fix -- taking more discriminator steps between generator steps -- restores parity between the generator and discriminator and improves results. Additionally, with the goal of restoring parity, we experiment with other modifications -- namely, large batch sizes and adaptive discriminator update frequency -- to improve discriminator training and see further improvements in generation quality. Our results demonstrate that on standard image synthesis benchmarks, DPSGD outperforms all alternative GAN privatization schemes. Code: https://github.com/alexbie98/dpgan-revisit.