LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy
This work provides a practical solution for privacy protection in federated learning, which is crucial for applications handling sensitive user data, though it appears incremental as it builds on existing local differential privacy methods.
The paper tackled the problem of achieving strict privacy guarantees in federated learning by addressing issues like information exposure, poor accuracy, and high dimensionality in local differential privacy mechanisms, resulting in superior performance on datasets like MNIST, Fashion-MNIST, and CIFAR-10 with strong privacy guarantees.
Train machine learning models on sensitive user data has raised increasing privacy concerns in many areas. Federated learning is a popular approach for privacy protection that collects the local gradient information instead of real data. One way to achieve a strict privacy guarantee is to apply local differential privacy into federated learning. However, previous works do not give a practical solution due to three issues. First, the noisy data is close to its original value with high probability, increasing the risk of information exposure. Second, a large variance is introduced to the estimated average, causing poor accuracy. Last, the privacy budget explodes due to the high dimensionality of weights in deep learning models. In this paper, we proposed a novel design of local differential privacy mechanism for federated learning to address the abovementioned issues. It is capable of making the data more distinct from its original value and introducing lower variance. Moreover, the proposed mechanism bypasses the curse of dimensionality by splitting and shuffling model updates. A series of empirical evaluations on three commonly used datasets, MNIST, Fashion-MNIST and CIFAR-10, demonstrate that our solution can not only achieve superior deep learning performance but also provide a strong privacy guarantee at the same time.