Generalized PTR: User-Friendly Recipes for Data-Adaptive Algorithms with Differential Privacy
This work addresses the challenge of designing differentially private algorithms that adapt to data characteristics, benefiting privacy-preserving machine learning practitioners.
The paper extends the Propose-Test-Release framework to handle data-dependent privacy losses, enabling its application to queries with unbounded sensitivity, and demonstrates this with private linear regression and solving an open problem in PATE for model release.
The ''Propose-Test-Release'' (PTR) framework is a classic recipe for designing differentially private (DP) algorithms that are data-adaptive, i.e. those that add less noise when the input dataset is nice. We extend PTR to a more general setting by privately testing data-dependent privacy losses rather than local sensitivity, hence making it applicable beyond the standard noise-adding mechanisms, e.g. to queries with unbounded or undefined sensitivity. We demonstrate the versatility of generalized PTR using private linear regression as a case study. Additionally, we apply our algorithm to solve an open problem from ''Private Aggregation of Teacher Ensembles (PATE)'' -- privately releasing the entire model with a delicate data-dependent analysis.