Evaluating Differentially Private Machine Learning in Practice
This work highlights critical limitations in real-world privacy protection for machine learning users, showing that current methods are often ineffective, which is an incremental but important finding for the privacy community.
The paper evaluates the practical privacy-utility trade-offs in differentially private machine learning, finding a large gap between guaranteed privacy loss bounds and measurable effective privacy loss, with current mechanisms failing to provide acceptable trade-offs for complex tasks.
Differential privacy is a strong notion for privacy that can be used to prove formal guarantees, in terms of a privacy budget, $ε$, about how much information is leaked by a mechanism. However, implementations of privacy-preserving machine learning often select large values of $ε$ in order to get acceptable utility of the model, with little understanding of the impact of such choices on meaningful privacy. Moreover, in scenarios where iterative learning procedures are used, differential privacy variants that offer tighter analyses are used which appear to reduce the needed privacy budget but present poorly understood trade-offs between privacy and utility. In this paper, we quantify the impact of these choices on privacy in experiments with logistic regression and neural network models. Our main finding is that there is a huge gap between the upper bounds on privacy loss that can be guaranteed, even with advanced mechanisms, and the effective privacy loss that can be measured using current inference attacks. Current mechanisms for differentially private machine learning rarely offer acceptable utility-privacy trade-offs with guarantees for complex learning tasks: settings that provide limited accuracy loss provide meaningless privacy guarantees, and settings that provide strong privacy guarantees result in useless models. Code for the experiments can be found here: https://github.com/bargavj/EvaluatingDPML