Optimal Accounting of Differential Privacy via Characteristic Function
This work addresses a fundamental challenge in differential privacy for applications like machine learning and federated learning, offering a novel unification of existing methods with tight guarantees.
The paper tackles the problem of privacy accounting for differential privacy compositions by proposing a unified approach using characteristic functions, which provides exactly tight privacy guarantees and can be converted to other privacy measures. The method demonstrates flexibility and tightness on various DP mechanisms, with experimental results showing improved accuracy in privacy loss estimates.
Characterizing the privacy degradation over compositions, i.e., privacy accounting, is a fundamental topic in differential privacy (DP) with many applications to differentially private machine learning and federated learning. We propose a unification of recent advances (Renyi DP, privacy profiles, $f$-DP and the PLD formalism) via the \emph{characteristic function} ($φ$-function) of a certain \emph{dominating} privacy loss random variable. We show that our approach allows \emph{natural} adaptive composition like Renyi DP, provides \emph{exactly tight} privacy accounting like PLD, and can be (often \emph{losslessly}) converted to privacy profile and $f$-DP, thus providing $(ε,δ)$-DP guarantees and interpretable tradeoff functions. Algorithmically, we propose an \emph{analytical Fourier accountant} that represents the \emph{complex} logarithm of $φ$-functions symbolically and uses Gaussian quadrature for numerical computation. On several popular DP mechanisms and their subsampled counterparts, we demonstrate the flexibility and tightness of our approach in theory and experiments.