Towards Distributed Privacy-Preserving Prediction
This addresses privacy concerns for parties in distributed machine learning settings, though it is incremental as it builds on existing techniques like differential privacy and homomorphic encryption.
The paper tackles the problem of privacy leakage in machine learning by proposing a Distributed Privacy-Preserving Prediction (DPPP) framework that combines predictions from multiple models without sharing sensitive data or parameters, achieving comparable performance to non-private methods and outperforming local differential privacy and standalone approaches.
In privacy-preserving machine learning, individual parties are reluctant to share their sensitive training data due to privacy concerns. Even the trained model parameters or prediction can pose serious privacy leakage. To address these problems, we demonstrate a generally applicable Distributed Privacy-Preserving Prediction (DPPP) framework, in which instead of sharing more sensitive data or model parameters, an untrusted aggregator combines only multiple models' predictions under provable privacy guarantee. Our framework integrates two main techniques to guarantee individual privacy. First, we introduce the improved Binomial Mechanism and Discrete Gaussian Mechanism to achieve distributed differential privacy. Second, we utilize homomorphic encryption to ensure that the aggregator learns nothing but the noisy aggregated prediction. Experimental results demonstrate that our framework has comparable performance to the non-private frameworks and delivers better results than the local differentially private framework and standalone framework.