Optimal Unbiased Randomizers for Regression with Label Differential Privacy
This work addresses the challenge of preserving privacy in regression tasks for machine learning practitioners, though it appears incremental as it builds on existing label DP methods with specific improvements.
The authors tackled the problem of training regression models with label differential privacy by proposing a new family of unbiased label randomizers that leverage bias-variance trade-offs and a privately estimated prior distribution over labels, achieving state-of-the-art privacy-utility trade-offs on several datasets.
We propose a new family of label randomizers for training regression models under the constraint of label differential privacy (DP). In particular, we leverage the trade-offs between bias and variance to construct better label randomizers depending on a privately estimated prior distribution over the labels. We demonstrate that these randomizers achieve state-of-the-art privacy-utility trade-offs on several datasets, highlighting the importance of reducing bias when training neural networks with label DP. We also provide theoretical results shedding light on the structural properties of the optimal unbiased randomizers.