DP-SIGNSGD: When Efficiency Meets Privacy and Robustness
This work addresses privacy concerns in federated learning for distributed parties, but it is incremental as it builds upon existing SIGNSGD methods.
The paper tackles the challenge of ensuring privacy in federated learning by proposing DP-SIGNSGD, which extends SIGNSGD to provide differential privacy guarantees while maintaining efficiency and robustness, with experimental validation on image datasets.
Federated learning (FL) has emerged as a promising collaboration paradigm by enabling a multitude of parties to construct a joint model without exposing their private training data. Three main challenges in FL are efficiency, privacy, and robustness. The recently proposed SIGNSGD with majority vote shows a promising direction to deal with efficiency and Byzantine robustness. However, there is no guarantee that SIGNSGD is privacy-preserving. In this paper, we bridge this gap by presenting an improved method called DP-SIGNSGD, which can meet all the aforementioned properties. We further propose an error-feedback variant of DP-SIGNSGD to improve accuracy. Experimental results on benchmark image datasets demonstrate the effectiveness of our proposed methods.