Privacy Amplification for the Gaussian Mechanism via Bounded Support
This work addresses the problem of providing tighter, data-dependent privacy guarantees for individuals in specific datasets, which is incremental but improves practical privacy in machine learning.
The paper tackled the lack of private mechanisms that fully leverage data-dependent privacy accounting by proposing modifications to the Gaussian mechanism with bounded support, showing that these amplify privacy guarantees and reduce the per-instance differential privacy bound by up to 30% in experiments without harming model utility.
Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. These guarantees can be desirable compared to vanilla DP in real world settings as they tightly upper-bound the privacy leakage for a $\textit{specific}$ individual in an $\textit{actual}$ dataset, rather than considering worst-case datasets. While these frameworks are beginning to gain popularity, to date, there is a lack of private mechanisms that can fully leverage advantages of data-dependent accounting. To bridge this gap, we propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting. Experiments on model training with DP-SGD show that using bounded support Gaussian mechanisms can provide a reduction of the pDP bound $ε$ by as much as 30% without negative effects on model utility.