Improved Localized Machine Unlearning Through the Lens of Memorization
This work addresses the need for efficient data removal in machine learning models, which is crucial for applications like handling outdated or poisoned data, though it appears incremental as it builds on existing unlearning methods with improved localization.
The paper tackles the problem of machine unlearning by focusing on localized unlearning, where only a subset of model parameters is modified, and introduces a new algorithm called Deletion by Example Localization (DEL) that sets a new state-of-the-art for unlearning metrics while maintaining high test accuracy.
Machine unlearning refers to removing the influence of a specified subset of training data from a machine learning model, efficiently, after it has already been trained. This is important for key applications, including making the model more accurate by removing outdated, mislabeled, or poisoned data. In this work, we study localized unlearning, where the unlearning algorithm operates on a (small) identified subset of parameters. Drawing inspiration from the memorization literature, we propose an improved localization strategy that yields strong results when paired with existing unlearning algorithms. We also propose a new unlearning algorithm, Deletion by Example Localization (DEL), that resets the parameters deemed-to-be most critical according to our localization strategy, and then finetunes them. Our extensive experiments on different datasets, forget sets and metrics reveal that DEL sets a new state-of-the-art for unlearning metrics, against both localized and full-parameter methods, while modifying a small subset of parameters, and outperforms the state-of-the-art localized unlearning in terms of test accuracy too.