Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection
This addresses data-privacy compliance needs for machine learning models, though it is an incremental improvement over existing unlearning methods.
The paper tackles the problem of machine unlearning by introducing Projected-Gradient Unlearning (PGU), which efficiently removes the effect of specific training samples from a deep neural network while preserving performance on the remaining dataset, achieving results comparable to retraining from scratch.
Recent data-privacy laws have sparked interest in machine unlearning, which involves removing the effect of specific training samples from a learnt model as if they were never present in the original training dataset. The challenge of machine unlearning is to discard information about the ``forget'' data in the learnt model without altering the knowledge about the remaining dataset and to do so more efficiently than the naive retraining approach. To achieve this, we adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU), in which the model takes steps in the orthogonal direction to the gradient subspaces deemed unimportant for the retaining dataset, so as to its knowledge is preserved. By utilizing Stochastic Gradient Descent (SGD) to update the model weights, our method can efficiently scale to any model and dataset size. We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible. Our code is available at https://github.com/hnanhtuan/projected_gradient_unlearning.