Gaussian Processes with Differential Privacy
This work addresses privacy concerns in GP-based prediction tasks, offering a more comprehensive DP solution, though it is incremental as it builds on sparse GP methodology.
The paper tackled the limitation of previous differential privacy (DP) methods for Gaussian processes (GPs) by introducing a method that protects both model inputs and outputs, achieving accurate models under strong privacy protection with sufficient data.
Gaussian processes (GPs) are non-parametric Bayesian models that are widely used for diverse prediction tasks. Previous work in adding strong privacy protection to GPs via differential privacy (DP) has been limited to protecting only the privacy of the prediction targets (model outputs) but not inputs. We break this limitation by introducing GPs with DP protection for both model inputs and outputs. We achieve this by using sparse GP methodology and publishing a private variational approximation on known inducing points. The approximation covariance is adjusted to approximately account for the added uncertainty from DP noise. The approximation can be used to compute arbitrary predictions using standard sparse GP techniques. We propose a method for hyperparameter learning using a private selection protocol applied to validation set log-likelihood. Our experiments demonstrate that given sufficient amount of data, the method can produce accurate models under strong privacy protection.