Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning
This addresses privacy risks for users in FL systems, offering a novel mitigation approach, though it is incremental in improving existing FL security.
The paper tackles the problem of deanonymization in Federated Learning by showing that model updates encode user-specific statistical signals, enabling adversaries to identify devices with high accuracy (e.g., up to 90% in some scenarios). It proposes data-augmentation strategies that reduce deanonymization risks by over 50% while maintaining model utility.
Federated Learning (FL) systems are gaining popularity as a solution to training Machine Learning (ML) models from large-scale user data collected on personal devices (e.g., smartphones) without their raw data leaving the device. At the core of FL is a network of anonymous user devices sharing training information (model parameter updates) computed locally on personal data. However, the type and degree to which user-specific information is encoded in the model updates is poorly understood. In this paper, we identify model updates encode subtle variations in which users capture and generate data. The variations provide a strong statistical signal, allowing an adversary to effectively deanonymize participating devices using a limited set of auxiliary data. We analyze resulting deanonymization attacks on diverse tasks on real-world (anonymized) user-generated data across a range of closed- and open-world scenarios. We study various strategies to mitigate the risks of deanonymization. As random perturbation methods do not offer convincing operating points, we propose data-augmentation strategies which introduces adversarial biases in device data and thereby, offer substantial protection against deanonymization threats with little effect on utility.