No Peek: A Survey of private distributed deep learning
This is an incremental survey that addresses the problem of data privacy in distributed deep learning for researchers and practitioners.
The paper surveys distributed deep learning methods that enable training or inference without accessing raw client data, aiming to protect confidential data patterns while allowing server-side model training, and compares approaches like federated learning, split learning, and privacy techniques such as differential privacy and homomorphic encryption.
We survey distributed deep learning models for training or inference without accessing raw data from clients. These methods aim to protect confidential patterns in data while still allowing servers to train models. The distributed deep learning methods of federated learning, split learning and large batch stochastic gradient descent are compared in addition to private and secure approaches of differential privacy, homomorphic encryption, oblivious transfer and garbled circuits in the context of neural networks. We study their benefits, limitations and trade-offs with regards to computational resources, data leakage and communication efficiency and also share our anticipated future trends.