Learning to Forget using Hypernetworks
This addresses the need for dynamic targeted data removal in machine learning models to enhance privacy and security, though it appears incremental as it builds on existing hypernetwork and diffusion model concepts.
The paper tackles the problem of machine unlearning to remove adversarial data poisoning and comply with privacy regulations by introducing HyperForget, a framework that uses hypernetworks to dynamically sample models that forget targeted data; in experiments, unlearned models achieved zero accuracy on the forget set while maintaining good accuracy on retain sets.
Machine unlearning is gaining increasing attention as a way to remove adversarial data poisoning attacks from already trained models and to comply with privacy and AI regulations. The objective is to unlearn the effect of undesired data from a trained model while maintaining performance on the remaining data. This paper introduces HyperForget, a novel machine unlearning framework that leverages hypernetworks - neural networks that generate parameters for other networks - to dynamically sample models that lack knowledge of targeted data while preserving essential capabilities. Leveraging diffusion models, we implement two Diffusion HyperForget Networks and used them to sample unlearned models in Proof-of-Concept experiments. The unlearned models obtained zero accuracy on the forget set, while preserving good accuracy on the retain sets, highlighting the potential of HyperForget for dynamic targeted data removal and a promising direction for developing adaptive machine unlearning algorithms.