LDP-Fed: Federated Learning with Local Differential Privacy
It addresses privacy concerns in federated learning for participants with sensitive data, though it appears incremental as it builds on existing LDP and federated learning concepts.
This paper tackles the challenge of applying local differential privacy (LDP) to federated learning, where existing LDP protocols are inapplicable due to high-dimensional, continuous model updates, and presents LDP-Fed, a system that achieves formal privacy guarantees while training deep neural networks on public data, with results showing competitive model accuracy and privacy preservation compared to state-of-the-art approaches.
This paper presents LDP-Fed, a novel federated learning system with a formal privacy guarantee using local differential privacy (LDP). Existing LDP protocols are developed primarily to ensure data privacy in the collection of single numerical or categorical values, such as click count in Web access logs. However, in federated learning model parameter updates are collected iteratively from each participant and consist of high dimensional, continuous values with high precision (10s of digits after the decimal point), making existing LDP protocols inapplicable. To address this challenge in LDP-Fed, we design and develop two novel approaches. First, LDP-Fed's LDP Module provides a formal differential privacy guarantee for the repeated collection of model training parameters in the federated training of large-scale neural networks over multiple individual participants' private datasets. Second, LDP-Fed implements a suite of selection and filtering techniques for perturbing and sharing select parameter updates with the parameter server. We validate our system deployed with a condensed LDP protocol in training deep neural networks on public data. We compare this version of LDP-Fed, coined CLDP-Fed, with other state-of-the-art approaches with respect to model accuracy, privacy preservation, and system capabilities.