Beyond Fixed Anchors: Precisely Erasing Concepts with Sibling Exclusive Counterparts
This addresses the issue of precise concept erasure for users of text-to-image models, representing an incremental improvement over existing methods.
The paper tackles the problem of concept re-emergence and erosion in text-to-image diffusion models by proposing SELECT, a dynamic anchor selection framework that replaces fixed anchors with sibling exclusive concepts, achieving efficient adaptation and outperforming baselines with an average of 4 seconds for anchor mining per concept.
Existing concept erasure methods for text-to-image diffusion models commonly rely on fixed anchor strategies, which often lead to critical issues such as concept re-emergence and erosion. To address this, we conduct causal tracing to reveal the inherent sensitivity of erasure to anchor selection and define Sibling Exclusive Concepts as a superior class of anchors. Based on this insight, we propose \textbf{SELECT} (Sibling-Exclusive Evaluation for Contextual Targeting), a dynamic anchor selection framework designed to overcome the limitations of fixed anchors. Our framework introduces a novel two-stage evaluation mechanism that automatically discovers optimal anchors for precise erasure while identifying critical boundary anchors to preserve related concepts. Extensive evaluations demonstrate that SELECT, as a universal anchor solution, not only efficiently adapts to multiple erasure frameworks but also consistently outperforms existing baselines across key performance metrics, averaging only 4 seconds for anchor mining of a single concept.