Rényi Differential Privacy of the Sampled Gaussian Mechanism
This work provides incremental improvements in understanding and computing privacy guarantees for SGM, benefiting researchers and practitioners in privacy-preserving machine learning.
The authors tackled the problem of precisely characterizing the privacy properties of the Sampled Gaussian Mechanism (SGM), a key tool in machine learning for differential privacy, and developed a numerically stable procedure to compute its Rényi Differential Privacy along with a nearly tight closed-form bound.
The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadratically, rather than linearly, with the sampling rate. Characterizing the precise privacy properties of SGM motivated development of several relaxations of the notion of differential privacy. This work unifies and fills in gaps in published results on SGM. We describe a numerically stable procedure for precise computation of SGM's Rényi Differential Privacy and prove a nearly tight (within a small constant factor) closed-form bound.