Improving the Privacy and Practicality of Objective Perturbation for Differentially Private Linear Learners
This work addresses the need for more private and practical methods in machine learning, particularly for linear learners, but it is incremental as it revamps an existing mechanism rather than introducing a new paradigm.
The paper tackled the problem of objective perturbation being less popular than DP-SGD for privacy-preserving machine learning by improving its privacy analysis and computational tools, resulting in competitive performance with DP-SGD on unconstrained convex generalized linear problems.
In the arena of privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) has outstripped the objective perturbation mechanism in popularity and interest. Though unrivaled in versatility, DP-SGD requires a non-trivial privacy overhead (for privately tuning the model's hyperparameters) and a computational complexity which might be extravagant for simple models such as linear and logistic regression. This paper revamps the objective perturbation mechanism with tighter privacy analyses and new computational tools that boost it to perform competitively with DP-SGD on unconstrained convex generalized linear problems.