Differential Private Stack Generalization with an Application to Diabetes Prediction
This work addresses privacy-preserving machine learning for sensitive data like medical records, offering an incremental improvement in differential privacy methods.
The paper tackles the performance degradation in differentially private logistic regression by proposing a stacking ensemble method with sample-based or feature-based partitioning, showing that feature-based partitioning requires fewer samples for the same privacy budget. It demonstrates effectiveness on benchmark datasets and applies it to cross-organizational diabetes prediction using the RUIJIN dataset.
To meet the standard of differential privacy, noise is usually added into the original data, which inevitably deteriorates the predicting performance of subsequent learning algorithms. In this paper, motivated by the success of improving predicting performance by ensemble learning, we propose to enhance privacy-preserving logistic regression by stacking. We show that this can be done either by sample-based or feature-based partitioning. However, we prove that when privacy-budgets are the same, feature-based partitioning requires fewer samples than sample-based one, and thus likely has better empirical performance. As transfer learning is difficult to be integrated with a differential privacy guarantee, we further combine the proposed method with hypothesis transfer learning to address the problem of learning across different organizations. Finally, we not only demonstrate the effectiveness of our method on two benchmark data sets, i.e., MNIST and NEWS20, but also apply it into a real application of cross-organizational diabetes prediction from RUIJIN data set, where privacy is of significant concern.