Gradient Masking and the Underestimated Robustness Threats of Differential Privacy in Deep Learning
This work addresses the overlooked interaction between privacy and security in deep learning, revealing critical vulnerabilities for practitioners using DP to protect sensitive data.
The paper investigates how training neural networks with Differential Privacy (DP) affects their robustness to adversarial attacks, finding that DP models are less robust and exhibit better transferability of adversarial examples among themselves, and that poor DP parameter choices can cause gradient masking, creating a false sense of security.
An important problem in deep learning is the privacy and security of neural networks (NNs). Both aspects have long been considered separately. To date, it is still poorly understood how privacy enhancing training affects the robustness of NNs. This paper experimentally evaluates the impact of training with Differential Privacy (DP), a standard method for privacy preservation, on model vulnerability against a broad range of adversarial attacks. The results suggest that private models are less robust than their non-private counterparts, and that adversarial examples transfer better among DP models than between non-private and private ones. Furthermore, detailed analyses of DP and non-DP models suggest significant differences between their gradients. Additionally, this work is the first to observe that an unfavorable choice of parameters in DP training can lead to gradient masking, and, thereby, results in a wrong sense of security.