PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training
This work addresses privacy risks in cloud-based DNN training for users with limited local resources, but it is incremental as it builds on existing split learning and privacy-preserving methods.
The authors tackled the problem of enabling cloud-based deep neural network training while preserving data privacy by splitting DNNs between local platforms and the cloud, using pre-trained local networks to generate intermediate representations. They validated this approach on a CNN-based image classification task with the CIFAR-10 dataset, optimizing accuracy under privacy and resource constraints.
Massive data exist among user local platforms that usually cannot support deep neural network (DNN) training due to computation and storage resource constraints. Cloud-based training schemes provide beneficial services but suffer from potential privacy risks due to excessive user data collection. To enable cloud-based DNN training while protecting the data privacy simultaneously, we propose to leverage the intermediate representations of the data, which is achieved by splitting the DNNs and deploying them separately onto local platforms and the cloud. The local neural network (NN) is used to generate the feature representations. To avoid local training and protect data privacy, the local NN is derived from pre-trained NNs. The cloud NN is then trained based on the extracted intermediate representations for the target learning task. We validate the idea of DNN splitting by characterizing the dependency of privacy loss and classification accuracy on the local NN topology for a convolutional NN (CNN) based image classification task. Based on the characterization, we further propose PrivyNet to determine the local NN topology, which optimizes the accuracy of the target learning task under the constraints on privacy loss, local computation, and storage. The efficiency and effectiveness of PrivyNet are demonstrated with the CIFAR-10 dataset.