Prompt-Guided Environmentally Consistent Adversarial Patch
This addresses security vulnerabilities in vision-based systems like facial recognition and autonomous driving by making adversarial attacks more stealthy and effective, representing a novel method for a known bottleneck.
The paper tackles the problem of adversarial patches being easily detectable and lacking environmental consistency by introducing PG-ECAP, which uses diffusion models and alignment losses to generate patches that blend seamlessly into environments, achieving higher attack success rates and improved consistency compared to existing methods.
Adversarial attacks in the physical world pose a significant threat to the security of vision-based systems, such as facial recognition and autonomous driving. Existing adversarial patch methods primarily focus on improving attack performance, but they often produce patches that are easily detectable by humans and struggle to achieve environmental consistency, i.e., blending patches into the environment. This paper introduces a novel approach for generating adversarial patches, which addresses both the visual naturalness and environmental consistency of the patches. We propose Prompt-Guided Environmentally Consistent Adversarial Patch (PG-ECAP), a method that aligns the patch with the environment to ensure seamless integration into the environment. The approach leverages diffusion models to generate patches that are both environmental consistency and effective in evading detection. To further enhance the naturalness and consistency, we introduce two alignment losses: Prompt Alignment Loss and Latent Space Alignment Loss, ensuring that the generated patch maintains its adversarial properties while fitting naturally within its environment. Extensive experiments in both digital and physical domains demonstrate that PG-ECAP outperforms existing methods in attack success rate and environmental consistency.