ForgetMe: Evaluating Selective Forgetting in Generative Models
This work addresses privacy concerns in generative AI by providing benchmarks for selective forgetting, though it is incremental as it builds on existing unlearning methods.
The paper tackles the challenge of selective unlearning in diffusion models for privacy compliance by proposing the ForgetMe dataset and Entangled evaluation metric, achieving validation through LoRA fine-tuning on Stable Diffusion across diverse datasets including CUB-200-2011, Stanford-Dogs, ImageNet, and synthetic cats.
The widespread adoption of diffusion models in image generation has increased the demand for privacy-compliant unlearning. However, due to the high-dimensional nature and complex feature representations of diffusion models, achieving selective unlearning remains challenging, as existing methods struggle to remove sensitive information while preserving the consistency of non-sensitive regions. To address this, we propose an Automatic Dataset Creation Framework based on prompt-based layered editing and training-free local feature removal, constructing the ForgetMe dataset and introducing the Entangled evaluation metric. The Entangled metric quantifies unlearning effectiveness by assessing the similarity and consistency between the target and background regions and supports both paired (Entangled-D) and unpaired (Entangled-S) image data, enabling unsupervised evaluation. The ForgetMe dataset encompasses a diverse set of real and synthetic scenarios, including CUB-200-2011 (Birds), Stanford-Dogs, ImageNet, and a synthetic cat dataset. We apply LoRA fine-tuning on Stable Diffusion to achieve selective unlearning on this dataset and validate the effectiveness of both the ForgetMe dataset and the Entangled metric, establishing them as benchmarks for selective unlearning. Our work provides a scalable and adaptable solution for advancing privacy-preserving generative AI.