Certified Unlearning for Neural Networks
This addresses privacy concerns and regulatory requirements like the 'right to be forgotten' for users and organizations, offering a broadly applicable solution with formal guarantees, though it builds on existing concepts in privacy amplification.
The paper tackles the problem of machine unlearning to remove specific training data from neural networks for privacy and regulatory compliance, proposing a method that uses noisy fine-tuning on retain data to achieve provable unlearning guarantees and outperforms existing baselines in practice.
We address the problem of machine unlearning, where the goal is to remove the influence of specific training data from a model upon request, motivated by privacy concerns and regulatory requirements such as the "right to be forgotten." Unfortunately, existing methods rely on restrictive assumptions or lack formal guarantees. To this end, we propose a novel method for certified machine unlearning, leveraging the connection between unlearning and privacy amplification by stochastic post-processing. Our method uses noisy fine-tuning on the retain data, i.e., data that does not need to be removed, to ensure provable unlearning guarantees. This approach requires no assumptions about the underlying loss function, making it broadly applicable across diverse settings. We analyze the theoretical trade-offs in efficiency and accuracy and demonstrate empirically that our method not only achieves formal unlearning guarantees but also performs effectively in practice, outperforming existing baselines. Our code is available at https://github.com/stair-lab/certified-unlearning-neural-networks-icml-2025