An Unlearning Framework for Continual Learning
This addresses safety and privacy concerns in AI by enabling unlearning in incremental learning systems, though it is incremental as it adapts existing unlearning concepts to a new paradigm.
The paper tackles the problem of applying machine unlearning in continual learning environments, where conventional methods cause performance degradation and task relapse, and proposes UnCLe, a data-free framework that uses a hypernetwork to unlearn tasks by aligning parameters with noise, achieving minimal disruption to retained knowledge in empirical evaluations.
Growing concerns surrounding AI safety and data privacy have driven the development of Machine Unlearning as a potential solution. However, current machine unlearning algorithms are designed to complement the offline training paradigm. The emergence of the Continual Learning (CL) paradigm promises incremental model updates, enabling models to learn new tasks sequentially. Naturally, some of those tasks may need to be unlearned to address safety or privacy concerns that might arise. We find that applying conventional unlearning algorithms in continual learning environments creates two critical problems: performance degradation on retained tasks and task relapse, where previously unlearned tasks resurface during subsequent learning. Furthermore, most unlearning algorithms require data to operate, which conflicts with CL's philosophy of discarding past data. A clear need arises for unlearning algorithms that are data-free and mindful of future learning. To that end, we propose UnCLe, an Unlearning framework for Continual Learning. UnCLe employs a hypernetwork that learns to generate task-specific network parameters, using task embeddings. Tasks are unlearned by aligning the corresponding generated network parameters with noise, without requiring any data. Empirical evaluations on several vision data sets demonstrate UnCLe's ability to sequentially perform multiple learning and unlearning operations with minimal disruption to previously acquired knowledge.