Fed-DPRoC:Communication-Efficient Differentially Private and Robust Federated Learning
This addresses the problem of secure and efficient federated learning for distributed systems, representing an incremental advancement by integrating existing techniques in a novel way.
The paper tackles the challenge of making federated learning simultaneously private, robust to malicious clients, and communication-efficient, achieving this through a novel framework called Fed-DPRoC that combines compression with robust aggregation, as validated by experiments on CIFAR-10 and Fashion MNIST showing improved robustness and utility.
We propose Fed-DPRoC, a novel federated learning framework that simultaneously ensures differential privacy (DP), Byzantine robustness, and communication efficiency. We introduce the concept of robust-compatible compression, which enables users to compress DP-protected updates while maintaining the robustness of the aggregation rule. We instantiate our framework as RobAJoL, combining the Johnson-Lindenstrauss (JL) transform for compression with robust averaging for robust aggregation. We theoretically prove the compatibility of JL transform with robust averaging and show that RobAJoL preserves robustness guarantees, ensures DP, and reduces communication cost. Experiments on CIFAR-10 and Fashion MNIST validate our theoretical claims and demonstrate that RobAJoL outperforms existing methods in terms of robustness and utility under different Byzantine attacks.