Privacy Amplification of Iterative Algorithms via Contraction Coefficients
This work addresses privacy concerns in machine learning for researchers and practitioners by providing improved theoretical bounds, though it is incremental as it builds on existing frameworks.
The paper tackles the problem of analyzing differential privacy guarantees for iterative algorithms by using contraction coefficients from information theory, specifically applying strong data processing inequalities for f-divergences to derive tighter bounds on privacy parameters for projected noisy stochastic gradient descent with hidden updates.
We investigate the framework of privacy amplification by iteration, recently proposed by Feldman et al., from an information-theoretic lens. We demonstrate that differential privacy guarantees of iterative mappings can be determined by a direct application of contraction coefficients derived from strong data processing inequalities for $f$-divergences. In particular, by generalizing the Dobrushin's contraction coefficient for total variation distance to an $f$-divergence known as $E_γ$-divergence, we derive tighter bounds on the differential privacy parameters of the projected noisy stochastic gradient descent algorithm with hidden intermediate updates.