Noise-Aware Differentially Private Variational Inference
This work addresses privacy-preserving inference for high-dimensional and non-conjugate models, but it is incremental as it extends existing noise-aware approaches to more complex scenarios.
The paper tackles the problem of unreliable results and biases in differentially private statistical inference by proposing a noise-aware approximate Bayesian inference method based on stochastic gradient variational inference, which achieves accurate coverages on high-dimensional Bayesian linear regression and well-calibrated predictive probabilities on Bayesian logistic regression with the UCI Adult dataset.
Differential privacy (DP) provides robust privacy guarantees for statistical inference, but this can lead to unreliable results and biases in downstream applications. While several noise-aware approaches have been proposed which integrate DP perturbation into the inference, they are limited to specific types of simple probabilistic models. In this work, we propose a novel method for noise-aware approximate Bayesian inference based on stochastic gradient variational inference which can also be applied to high-dimensional and non-conjugate models. We also propose a more accurate evaluation method for noise-aware posteriors. Empirically, our inference method has similar performance to existing methods in the domain where they are applicable. Outside this domain, we obtain accurate coverages on high-dimensional Bayesian linear regression and well-calibrated predictive probabilities on Bayesian logistic regression with the UCI Adult dataset.