Differential Privacy in Two-Layer Networks: How DP-SGD Harms Fairness and Robustness
This work provides theoretical insights into the performance degradation, fairness issues, and reduced adversarial robustness observed when applying differential privacy to neural networks, which is a problem for practitioners using sensitive data.
This paper theoretically investigates how differentially private stochastic gradient descent (DP-SGD) affects two-layer ReLU convolutional neural networks, finding that the noise required for privacy leads to suboptimal feature learning. Specifically, it shows that imbalanced feature-to-noise ratios (FNRs) cause disparate impact, noise disproportionately harms long-tailed data, and noise injection increases vulnerability to adversarial attacks.
Differentially private learning is essential for training models on sensitive data, but empirical studies consistently show that it can degrade performance, introduce fairness issues like disparate impact, and reduce adversarial robustness. The theoretical underpinnings of these phenomena in modern, non-convex neural networks remain largely unexplored. This paper introduces a unified feature-centric framework to analyze the feature learning dynamics of differentially private stochastic gradient descent (DP-SGD) in two-layer ReLU convolutional neural networks. Our analysis establishes test loss bounds governed by a crucial metric: the feature-to-noise ratio (FNR). We demonstrate that the noise required for privacy leads to suboptimal feature learning, and specifically show that: 1) imbalanced FNRs across classes and subpopulations cause disparate impact; 2) even in the same class, noise has a greater negative impact on semantically long-tailed data; and 3) noise injection exacerbates vulnerability to adversarial attacks. Furthermore, our analysis reveals that the popular paradigm of public pre-training and private fine-tuning does not guarantee improvement, particularly under significant feature distribution shifts between datasets. Experiments on synthetic and real-world data corroborate our theoretical findings.