Considerations on the Theory of Training Models with Differential Privacy
This is an incremental book chapter offering intuitive explanations for researchers interested in privacy-preserving machine learning.
The paper provides a high-level overview of differential privacy in federated learning, focusing on its framework and properties, but does not present new results or concrete numbers.
In federated learning collaborative learning takes place by a set of clients who each want to remain in control of how their local training data is used, in particular, how can each client's local training data remain private? Differential privacy is one method to limit privacy leakage. We provide a general overview of its framework and provable properties, adopt the more recent hypothesis based definition called Gaussian DP or $f$-DP, and discuss Differentially Private Stochastic Gradient Descent (DP-SGD). We stay at a meta level and attempt intuitive explanations and insights \textit{in this book chapter}.