Efficient Skip Connections Realization for Secure Inference on Encrypted Data
This work addresses a bottleneck in privacy-preserving machine learning for applications like secure classification, though it is incremental as it optimizes an existing method.
The paper tackled the high computational cost of skip connections in deep learning models when performing secure inference on encrypted data using Homomorphic Encryption, achieving a 1.3x improvement in computing power while maintaining the same accuracy.
Homomorphic Encryption (HE) is a cryptographic tool that allows performing computation under encryption, which is used by many privacy-preserving machine learning solutions, for example, to perform secure classification. Modern deep learning applications yield good performance for example in image processing tasks benchmarks by including many skip connections. The latter appears to be very costly when attempting to execute model inference under HE. In this paper, we show that by replacing (mid-term) skip connections with (short-term) Dirac parameterization and (long-term) shared-source skip connection we were able to reduce the skip connections burden for HE-based solutions, achieving x1.3 computing power improvement for the same accuracy.