Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation
This addresses the problem of social harm from fake or disturbing AI-generated images, offering a universal defense solution.
The paper tackles the misuse of diffusion models for generating harmful content by introducing Anti-Reference, a method that adds imperceptible adversarial noise to images to defend against reference-based generation techniques, achieving effective protection against various models and commercial APIs.
Diffusion models have revolutionized generative modeling with their exceptional ability to produce high-fidelity images. However, misuse of such potent tools can lead to the creation of fake news or disturbing content targeting individuals, resulting in significant social harm. In this paper, we introduce Anti-Reference, a novel method that protects images from the threats posed by reference-based generation techniques by adding imperceptible adversarial noise to the images. We propose a unified loss function that enables joint attacks on fine-tuning-based customization methods, non-fine-tuning customization methods, and human-centric driving methods. Based on this loss, we train a Adversarial Noise Encoder to predict the noise or directly optimize the noise using the PGD method. Our method shows certain transfer attack capabilities, effectively challenging both gray-box models and some commercial APIs. Extensive experiments validate the performance of Anti-Reference, establishing a new benchmark in image security.