Learning Privately from Multiparty Data
This addresses the challenge of privacy-preserving machine learning for applications like crowdsensing, where data is distributed across many parties, though it is incremental as it builds on existing ensemble and differential privacy methods.
The paper tackles the problem of learning a differentially private global classifier from multiple parties' private data without accessing the raw data, by transferring knowledge from local classifiers using auxiliary unlabeled data and a risk-weighted method. The result is a generalization error bounded by O(ε^{-2}M^{-2}), enabling strong privacy without performance loss when the number of parties is large, as demonstrated in tasks like activity recognition and intrusion detection.
Learning a classifier from private data collected by multiple parties is an important problem that has many potential applications. How can we build an accurate and differentially private global classifier by combining locally-trained classifiers from different parties, without access to any party's private data? We propose to transfer the `knowledge' of the local classifier ensemble by first creating labeled data from auxiliary unlabeled data, and then train a global $ε$-differentially private classifier. We show that majority voting is too sensitive and therefore propose a new risk weighted by class probabilities estimated from the ensemble. Relative to a non-private solution, our private solution has a generalization error bounded by $O(ε^{-2}M^{-2})$ where $M$ is the number of parties. This allows strong privacy without performance loss when $M$ is large, such as in crowdsensing applications. We demonstrate the performance of our method with realistic tasks of activity recognition, network intrusion detection, and malicious URL detection.