Robustness Threats of Differential Privacy
This work identifies a potential vulnerability for users deploying differentially private models, specifically concerning their robustness to adversarial attacks and corruptions.
This paper investigates the relationship between differential privacy (DP) and the robustness of neural networks, finding an empirical trade-off. They demonstrate that DP-trained networks can be more vulnerable to various input perturbations compared to their non-private counterparts.
Differential privacy (DP) is a gold-standard concept of measuring and guaranteeing privacy in data analysis. It is well-known that the cost of adding DP to deep learning model is its accuracy. However, it remains unclear how it affects robustness of the model. Standard neural networks are not robust to different input perturbations: either adversarial attacks or common corruptions. In this paper, we empirically observe an interesting trade-off between privacy and robustness of neural networks. We experimentally demonstrate that networks, trained with DP, in some settings might be even more vulnerable in comparison to non-private versions. To explore this, we extensively study different robustness measurements, including FGSM and PGD adversaries, distance to linear decision boundaries, curvature profile, and performance on a corrupted dataset. Finally, we study how the main ingredients of differentially private neural networks training, such as gradient clipping and noise addition, affect (decrease and increase) the robustness of the model.