Differentially Private Bayesian Linear Regression
This work addresses uncertainty quantification for sensitive data in fields like healthcare or social sciences, representing an incremental improvement over existing differentially private point estimates.
The paper tackles the problem of performing Bayesian linear regression with differential privacy, showing that naive methods produce poor uncertainty estimates, and develops noise-aware methods that yield correct posterior distributions across various scenarios.
Linear regression is an important tool across many fields that work with sensitive human-sourced data. Significant prior work has focused on producing differentially private point estimates, which provide a privacy guarantee to individuals while still allowing modelers to draw insights from data by estimating regression coefficients. We investigate the problem of Bayesian linear regression, with the goal of computing posterior distributions that correctly quantify uncertainty given privately released statistics. We show that a naive approach that ignores the noise injected by the privacy mechanism does a poor job in realistic data settings. We then develop noise-aware methods that perform inference over the privacy mechanism and produce correct posteriors across a wide range of scenarios.