Reference-Guided Machine Unlearning
This addresses the need for stable and effective unlearning in machine learning models, though it appears incremental as it builds on existing approximate unlearning methods.
The paper tackles the problem of machine unlearning by proposing Reference-Guided Unlearning (ReGUn), which uses a held-out dataset to align model behavior on forget data with unseen data, resulting in consistently superior forgetting-utility trade-offs across various architectures and datasets.
Machine unlearning aims to remove the influence of specific data from trained models while preserving general utility. Existing approximate unlearning methods often rely on performance-degradation heuristics, such as loss maximization or random labeling. However, these signals can be poorly conditioned, leading to unstable optimization and harming the model's generalization. We argue that unlearning should instead prioritize distributional indistinguishability, aligning the model's behavior on forget data with its behavior on truly unseen data. Motivated by this, we propose Reference-Guided Unlearning (ReGUn), a framework that leverages a disjoint held-out dataset to provide a principled, class-conditioned reference for distillation. We demonstrate across various model architectures, natural image datasets, and varying forget fractions that ReGUn consistently outperforms standard approximate baselines, achieving a superior forgetting-utility trade-off.