AdaCliP: Adaptive Clipping for Private SGD
This work addresses privacy concerns in machine learning for users with sensitive data, offering an incremental improvement over existing differentially private SGD algorithms.
The paper tackles the problem of training machine learning models with differential privacy by proposing AdaCliP, an adaptive clipping method for private SGD that reduces added noise and improves model accuracy compared to previous methods.
Privacy preserving machine learning algorithms are crucial for learning models over user data to protect sensitive information. Motivated by this, differentially private stochastic gradient descent (SGD) algorithms for training machine learning models have been proposed. At each step, these algorithms modify the gradients and add noise proportional to the sensitivity of the modified gradients. Under this framework, we propose AdaCliP, a theoretically motivated differentially private SGD algorithm that provably adds less noise compared to the previous methods, by using coordinate-wise adaptive clipping of the gradient. We empirically demonstrate that AdaCliP reduces the amount of added noise and produces models with better accuracy.