Co-occurring associated retained concepts in Diffusion Unlearning
For practitioners using diffusion models, this work provides a method to remove harmful content without degrading related benign concepts, addressing a key limitation of existing unlearning approaches.
The paper addresses the problem of unintentionally suppressing benign co-occurring concepts (CARE) during diffusion model unlearning, and proposes ReCARE, a framework that explicitly safeguards these concepts while erasing only the target concept, achieving state-of-the-art performance in balancing erasure, utility, and CARE preservation.
Unlearning has emerged as a key technique to mitigate harmful content generation in diffusion models. However, existing methods often remove not only the target concept, but also benign co-occurring concepts. As illustrated in Fig.1, unlearning nudity can unintentionally suppress the concept of person, preventing a model from generating images with person. We define these undesirably suppressed co-occurring concepts that must be preserved CARE (Co-occurring Associated REtained concepts). Then, we introduce the CARE score, a general metric that directly quantifies their preservation across unlearning tasks. With this foundation, we propose ReCARE (Robust erasure for CARE), a framework that explicitly safeguards CARE while erasing only the target concept. ReCARE automatically constructs the CARE-set, a curated vocabulary of benign co-occurring tokens extracted from target images, and leverages this vocabulary during training for stable unlearning. Extensive experiments across various target concepts (Nudity, Van Gogh style, and Tench object) demonstrate that ReCARE achieves overall state-of-the-art performance in balancing robust concept erasure, overall utility, and CARE preservation.