Ablating Concepts in Text-to-Image Diffusion Models
This addresses copyright and ethical issues for users and developers of generative AI models, though it is an incremental improvement on existing model editing techniques.
The paper tackles the problem of removing copyrighted or unwanted concepts from pretrained text-to-image diffusion models without retraining, proposing an ablation method that prevents generation of target concepts while preserving related ones.
Large-scale text-to-image diffusion models can generate high-fidelity images with powerful compositional ability. However, these models are typically trained on an enormous amount of Internet data, often containing copyrighted material, licensed images, and personal photos. Furthermore, they have been found to replicate the style of various living artists or memorize exact training samples. How can we remove such copyrighted concepts or images without retraining the model from scratch? To achieve this goal, we propose an efficient method of ablating concepts in the pretrained model, i.e., preventing the generation of a target concept. Our algorithm learns to match the image distribution for a target style, instance, or text prompt we wish to ablate to the distribution corresponding to an anchor concept. This prevents the model from generating target concepts given its text condition. Extensive experiments show that our method can successfully prevent the generation of the ablated concept while preserving closely related concepts in the model.