Safeguarding Facial Identity against Diffusion-based Face Swapping via Cascading Pathway Disruption
This work addresses privacy and identity security concerns for individuals vulnerable to face swapping attacks, representing an incremental improvement over prior defenses.
The paper tackles the problem of defending against diffusion-based face swapping to protect privacy and identity security by proposing VoidFace, a method that disrupts the face swapping pipeline through targeted perturbations, achieving superior performance over existing defenses while maintaining high visual quality in adversarial faces.
The rapid evolution of diffusion models has democratized face swapping but also raises concerns about privacy and identity security. Existing proactive defenses, often adapted from image editing attacks, prove ineffective in this context. We attribute this failure to an oversight of the structural resilience and the unique static conditional guidance mechanism inherent in face swapping systems. To address this, we propose VoidFace, a systemic defense method that views face swapping as a coupled identity pathway. By injecting perturbations at critical bottlenecks, VoidFace induces cascading disruption throughout the pipeline. Specifically, we first introduce localization disruption and identity erasure to degrade physical regression and semantic embeddings, thereby impairing the accurate modeling of the source face. We then intervene in the generative domain by decoupling attention mechanisms to sever identity injection, and corrupting intermediate diffusion features to prevent the reconstruction of source identity. To ensure visual imperceptibility, we perform adversarial search in the latent manifold, guided by a perceptual adaptive strategy to balance attack potency with image quality. Extensive experiments show that VoidFace outperforms existing defenses across various diffusion-based swapping models, while producing adversarial faces with superior visual quality.