Distributed Differentially Private Computation of Functions with Correlated Noise
This work addresses privacy concerns in collaborative machine learning across multiple sites, offering a solution for scenarios like human health research, though it is incremental in improving existing methods.
The paper tackles the problem of poor accuracy in differentially private decentralized algorithms when datasets are small by proposing the CAPE framework, which achieves the same performance as centralized algorithms in decentralized settings, as shown by empirical results on regression and neural network problems.
Many applications of machine learning, such as human health research, involve processing private or sensitive information. Privacy concerns may impose significant hurdles to collaboration in scenarios where there are multiple sites holding data and the goal is to estimate properties jointly across all datasets. Differentially private decentralized algorithms can provide strong privacy guarantees. However, the accuracy of the joint estimates may be poor when the datasets at each site are small. This paper proposes a new framework, Correlation Assisted Private Estimation (CAPE), for designing privacy-preserving decentralized algorithms with better accuracy guarantees in an honest-but-curious model. CAPE can be used in conjunction with the functional mechanism for statistical and machine learning optimization problems. A tighter characterization of the functional mechanism is provided that allows CAPE to achieve the same performance as a centralized algorithm in the decentralized setting using all datasets. Empirical results on regression and neural network problems for both synthetic and real datasets show that differentially private methods can be competitive with non-private algorithms in many scenarios of interest.