Rethinking Machine Unlearning: Models Designed to Forget via Key Deletion
This addresses the practical need for efficient unlearning in deployment scenarios under privacy regulations, offering a novel approach rather than incremental improvements.
The paper tackles the problem of machine unlearning by proposing a new paradigm where models are designed to inherently support forgetting, rather than using post-hoc methods that require full data access and parameter updates. The result is MUNKEY, a memory-augmented transformer that enables zero-shot forgetting by deleting instance-identifying keys, outperforming all post-hoc baselines across natural image, fine-grained recognition, and medical datasets.
Machine unlearning is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training samples. Despite this, most existing methods tackle the problem purely from a post-hoc perspective. They attempt to erase the influence of targeted training samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We propose \textit{unlearning by design}, a novel paradigm in which models are directly trained to support forgetting as an inherent capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling direct zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.