Debugging Differential Privacy: A Case Study for Privacy Auditing
This work addresses the critical issue of ensuring privacy protection in AI systems for users and developers, though it is incremental as it applies existing auditing methods to a specific case.
The paper tackled the problem of verifying differential privacy guarantees in machine learning implementations by auditing an open-source differentially private deep learning algorithm, finding with 99.99999999% confidence that it did not satisfy the claimed privacy guarantee.
Differential Privacy can provide provable privacy guarantees for training data in machine learning. However, the presence of proofs does not preclude the presence of errors. Inspired by recent advances in auditing which have been used for estimating lower bounds on differentially private algorithms, here we show that auditing can also be used to find flaws in (purportedly) differentially private schemes. In this case study, we audit a recent open source implementation of a differentially private deep learning algorithm and find, with 99.99999999% confidence, that the implementation does not satisfy the claimed differential privacy guarantee.