Towards Scalable and Privacy-Preserving Deep Neural Network via Algorithmic-Cryptographic Co-design
This work aims to enable scalable and privacy-preserving deep neural network learning for organizations dealing with sensitive data, addressing the trade-off between privacy and scalability.
The paper addresses the challenge of data isolation in Deep Neural Networks by proposing SPNN, a framework that combines algorithmic and cryptographic techniques. It splits DNN computations, with private data computations handled by data holders using secret sharing and homomorphic encryption, and heavy computations delegated to a server. The framework is implemented in a decentralized setting with user-friendly APIs and shows superior performance on real-world datasets.
Deep Neural Networks (DNNs) have achieved remarkable progress in various real-world applications, especially when abundant training data are provided. However, data isolation has become a serious problem currently. Existing works build privacy preserving DNN models from either algorithmic perspective or cryptographic perspective. The former mainly splits the DNN computation graph between data holders or between data holders and server, which demonstrates good scalability but suffers from accuracy loss and potential privacy risks. In contrast, the latter leverages time-consuming cryptographic techniques, which has strong privacy guarantee but poor scalability. In this paper, we propose SPNN - a Scalable and Privacy-preserving deep Neural Network learning framework, from algorithmic-cryptographic co-perspective. From algorithmic perspective, we split the computation graph of DNN models into two parts, i.e., the private data related computations that are performed by data holders and the rest heavy computations that are delegated to a server with high computation ability. From cryptographic perspective, we propose using two types of cryptographic techniques, i.e., secret sharing and homomorphic encryption, for the isolated data holders to conduct private data related computations privately and cooperatively. Furthermore, we implement SPNN in a decentralized setting and introduce user-friendly APIs. Experimental results conducted on real-world datasets demonstrate the superiority of SPNN.