Architecture Matters: Investigating the Influence of Differential Privacy on Neural Network Design
This addresses the problem of accuracy loss in differentially private neural networks for researchers and practitioners, but it is incremental as it highlights a need for future research without providing new solutions.
The study investigated how neural network architectures affect model accuracy under differential privacy constraints, finding that architectures performing well without privacy do not necessarily do so with privacy, indicating a gap in existing design knowledge.
One barrier to more widespread adoption of differentially private neural networks is the entailed accuracy loss. To address this issue, the relationship between neural network architectures and model accuracy under differential privacy constraints needs to be better understood. As a first step, we test whether extant knowledge on architecture design also holds in the differentially private setting. Our findings show that it does not; architectures that perform well without differential privacy, do not necessarily do so with differential privacy. Consequently, extant knowledge on neural network architecture design cannot be seamlessly translated into the differential privacy context. Future research is required to better understand the relationship between neural network architectures and model accuracy to enable better architecture design choices under differential privacy constraints.