Combining Stochastic Defenses to Resist Gradient Inversion: An Ablation Study
This work addresses privacy risks in federated learning for practitioners by showing that incremental combinations of existing defenses are necessary to achieve robust protection.
The paper tackles the vulnerability of individual stochastic defenses against gradient inversion attacks in federated learning, demonstrating that combining differential privacy and a stochastic privacy module reduces the attack success rate from 100% to 0% while maintaining model utility.
Gradient Inversion (GI) attacks are a ubiquitous threat in Federated Learning (FL) as they exploit gradient leakage to reconstruct supposedly private training data. Common defense mechanisms such as Differential Privacy (DP) or stochastic Privacy Modules (PMs) introduce randomness during gradient computation to prevent such attacks. However, we pose that if an attacker effectively mimics a client's stochastic gradient computation, the attacker can circumvent the defense and reconstruct clients' private training data. This paper introduces several targeted GI attacks that leverage this principle to bypass common defense mechanisms. As a result, we demonstrate that no individual defense provides sufficient privacy protection. To address this issue, we propose to combine multiple defenses. We conduct an extensive ablation study to evaluate the influence of various combinations of defenses on privacy protection and model utility. We observe that only the combination of DP and a stochastic PM was sufficient to decrease the Attack Success Rate (ASR) from 100% to 0%, thus preserving privacy. Moreover, we found that this combination of defenses consistently achieves the best trade-off between privacy and model utility.