zPROBE: Zero Peek Robustness Checks for Federated Learning
This work addresses the challenge of securing federated learning against malicious users while maintaining privacy, representing a novel integration of robustness checks with privacy-preserving techniques.
The paper tackles the problem of Byzantine attacks in privacy-preserving federated learning by introducing zPROBE, a framework that uses zero-knowledge proofs and randomized clustering to detect and remove malicious updates without revealing private user data, achieving low overhead in empirical evaluations.
Privacy-preserving federated learning allows multiple users to jointly train a model with coordination of a central server. The server only learns the final aggregation result, thus the users' (private) training data is not leaked from the individual model updates. However, keeping the individual updates private allows malicious users to perform Byzantine attacks and degrade the accuracy without being detected. Best existing defenses against Byzantine workers rely on robust rank-based statistics, e.g., median, to find malicious updates. However, implementing privacy-preserving rank-based statistics is nontrivial and not scalable in the secure domain, as it requires sorting all individual updates. We establish the first private robustness check that uses high break point rank-based statistics on aggregated model updates. By exploiting randomized clustering, we significantly improve the scalability of our defense without compromising privacy. We leverage our statistical bounds in zero-knowledge proofs to detect and remove malicious updates without revealing the private user updates. Our novel framework, zPROBE, enables Byzantine resilient and secure federated learning. Empirical evaluations demonstrate that zPROBE provides a low overhead solution to defend against state-of-the-art Byzantine attacks while preserving privacy.