MYSTIKO : : Cloud-Mediated, Private, Federated Gradient Descent
This work provides a method for privacy-preserving federated learning for organizations and individuals concerned about data leakage from gradient sharing, offering a solution that maintains model utility.
This paper addresses the privacy concerns in federated learning where sharing gradients can lead to reverse engineering attacks. The authors propose using additive homomorphic encryption (Paillier cipher) to secure federated gradient descent, ensuring privacy without sacrificing model accuracy or requiring hyperparameter tuning.
Federated learning enables multiple, distributed participants (potentially on different clouds) to collaborate and train machine/deep learning models by sharing parameters/gradients. However, sharing gradients, instead of centralizing data, may not be as private as one would expect. Reverse engineering attacks on plaintext gradients have been demonstrated to be practically feasible. Existing solutions for differentially private federated learning, while promising, lead to less accurate models and require nontrivial hyperparameter tuning. In this paper, we examine the use of additive homomorphic encryption (specifically the Paillier cipher) to design secure federated gradient descent techniques that (i) do not require addition of statistical noise or hyperparameter tuning, (ii) does not alter the final accuracy or utility of the final model, (iii) ensure that the plaintext model parameters/gradients of a participant are never revealed to any other participant or third party coordinator involved in the federated learning job, (iv) minimize the trust placed in any third party coordinator and (v) are efficient, with minimal overhead, and cost effective.