Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise
This work addresses privacy protection in federated learning for applications with sensitive data, but it is incremental as it builds on existing differential privacy methods by adding adaptive mechanisms.
The paper tackles the problem of privacy leakage in federated learning by proposing a differentially private scheme with adaptive noise, which improves learning performance by adapting gradient clipping and noise addition based on heterogeneity and convergence, achieving significant gains over previous methods in experiments on real-world datasets.
Federated learning seeks to address the issue of isolated data islands by making clients disclose only their local training models. However, it was demonstrated that private information could still be inferred by analyzing local model parameters, such as deep neural network model weights. Recently, differential privacy has been applied to federated learning to protect data privacy, but the noise added may degrade the learning performance much. Typically, in previous work, training parameters were clipped equally and noises were added uniformly. The heterogeneity and convergence of training parameters were simply not considered. In this paper, we propose a differentially private scheme for federated learning with adaptive noise (Adap DP-FL). Specifically, due to the gradient heterogeneity, we conduct adaptive gradient clipping for different clients and different rounds; due to the gradient convergence, we add decreasing noises accordingly. Extensive experiments on real-world datasets demonstrate that our Adap DP-FL outperforms previous methods significantly.