Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models
This reveals a critical vulnerability in federated learning systems that compromises user privacy, even in realistic training regimes previously considered secure.
The authors tackled the problem of data privacy in federated learning by introducing a new threat model where malicious modifications to the model architecture allow a server to directly obtain verbatim copies of user data from gradient updates, even for large aggregated batches where previous methods fail.
Federated learning has quickly gained popularity with its promises of increased user privacy and efficiency. Previous works have shown that federated gradient updates contain information that can be used to approximately recover user data in some situations. These previous attacks on user privacy have been limited in scope and do not scale to gradient updates aggregated over even a handful of data points, leaving some to conclude that data privacy is still intact for realistic training regimes. In this work, we introduce a new threat model based on minimal but malicious modifications of the shared model architecture which enable the server to directly obtain a verbatim copy of user data from gradient updates without solving difficult inverse problems. Even user data aggregated over large batches -- where previous methods fail to extract meaningful content -- can be reconstructed by these minimally modified models.