Noise-Aware Differentially Private Regression via Meta-Learning
This work addresses privacy-preserving machine learning for applications requiring accurate and calibrated predictions, representing an incremental improvement over existing DP methods.
The paper tackles the problem of maintaining model performance while ensuring differential privacy in high-stakes applications by introducing DPConvCNP, a meta-learning model that combines ConvCNP with an improved DP mechanism. It outperforms a DP Gaussian Process baseline, particularly on non-Gaussian data, with faster test times and less tuning required.
Many high-stakes applications require machine learning models that protect user privacy and provide well-calibrated, accurate predictions. While Differential Privacy (DP) is the gold standard for protecting user privacy, standard DP mechanisms typically significantly impair performance. One approach to mitigating this issue is pre-training models on simulated data before DP learning on the private data. In this work we go a step further, using simulated data to train a meta-learning model that combines the Convolutional Conditional Neural Process (ConvCNP) with an improved functional DP mechanism of Hall et al. [2013] yielding the DPConvCNP. DPConvCNP learns from simulated data how to map private data to a DP predictive model in one forward pass, and then provides accurate, well-calibrated predictions. We compare DPConvCNP with a DP Gaussian Process (GP) baseline with carefully tuned hyperparameters. The DPConvCNP outperforms the GP baseline, especially on non-Gaussian data, yet is much faster at test time and requires less tuning.