Three Tools for Practical Differential Privacy
This work addresses practical issues for practitioners implementing differential privacy in machine learning, but it is incremental as it builds on existing methods.
The paper tackled challenges in differentially private machine learning on real-world data by introducing three tools: sanity checks, an adaptive clipping bound, and large-batch training, resulting in improved practicality and model performance.
Differentially private learning on real-world data poses challenges for standard machine learning practice: privacy guarantees are difficult to interpret, hyperparameter tuning on private data reduces the privacy budget, and ad-hoc privacy attacks are often required to test model privacy. We introduce three tools to make differentially private machine learning more practical: (1) simple sanity checks which can be carried out in a centralized manner before training, (2) an adaptive clipping bound to reduce the effective number of tuneable privacy parameters, and (3) we show that large-batch training improves model performance.