Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers
This addresses the need for safe and controllable AI image generation by enabling targeted concept removal without affecting other capabilities, though it is incremental in improving erasure reliability.
The paper tackled the problem of reliably erasing specific concepts from text-to-image diffusion models to prevent generation of related images, achieving superior performance over previous methods in experiments.
Concept erasure in text-to-image diffusion models aims to disable pre-trained diffusion models from generating images related to a target concept. To perform reliable concept erasure, the properties of robustness and locality are desirable. The former refrains the model from producing images associated with the target concept for any paraphrased or learned prompts, while the latter preserves its ability in generating images with non-target concepts. In this paper, we propose Reliable Concept Erasing via Lightweight Erasers (Receler). It learns a lightweight Eraser to perform concept erasing while satisfying the above desirable properties through the proposed concept-localized regularization and adversarial prompt learning scheme. Experiments with various concepts verify the superiority of Receler over previous methods.