Certified Machine Unlearning via Noisy Stochastic Gradient Descent
This addresses the need for practical machine unlearning to support data privacy rights, offering a more efficient solution than existing methods.
The paper tackles the problem of efficiently removing the influence of specific data points from trained machine learning models to comply with privacy laws, achieving a similar utility with only 2% and 10% of gradient computations compared to state-of-the-art methods for mini-batch and full-batch settings, respectively.
``The right to be forgotten'' ensured by laws for user data privacy becomes increasingly important. Machine unlearning aims to efficiently remove the effect of certain data points on the trained model parameters so that it can be approximately the same as if one retrains the model from scratch. We propose to leverage projected noisy stochastic gradient descent for unlearning and establish its first approximate unlearning guarantee under the convexity assumption. Our approach exhibits several benefits, including provable complexity saving compared to retraining, and supporting sequential and batch unlearning. Both of these benefits are closely related to our new results on the infinite Wasserstein distance tracking of the adjacent (un)learning processes. Extensive experiments show that our approach achieves a similar utility under the same privacy constraint while using $2\%$ and $10\%$ of the gradient computations compared with the state-of-the-art gradient-based approximate unlearning methods for mini-batch and full-batch settings, respectively.