PolyJuice Makes It Real: Black-Box, Universal Red Teaming for Synthetic Image Detectors
This addresses the need for robust defenses against synthetic images in security and content moderation, offering a novel approach to red-teaming without white-box access, though it is incremental in improving existing red-teaming methods.
The paper tackled the problem of synthetic image detectors (SIDs) being vulnerable to attacks, proposing PolyJuice, a black-box and image-agnostic red-teaming method that exploits distribution shifts in latent space to deceive SIDs, achieving up to 84% success in attacks and improving detector performance by up to 30% when tuned on augmented datasets.
Synthetic image detectors (SIDs) are a key defense against the risks posed by the growing realism of images from text-to-image (T2I) models. Red teaming improves SID's effectiveness by identifying and exploiting their failure modes via misclassified synthetic images. However, existing red-teaming solutions (i) require white-box access to SIDs, which is infeasible for proprietary state-of-the-art detectors, and (ii) generate image-specific attacks through expensive online optimization. To address these limitations, we propose PolyJuice, the first black-box, image-agnostic red-teaming method for SIDs, based on an observed distribution shift in the T2I latent space between samples correctly and incorrectly classified by the SID. PolyJuice generates attacks by (i) identifying the direction of this shift through a lightweight offline process that only requires black-box access to the SID, and (ii) exploiting this direction by universally steering all generated images towards the SID's failure modes. PolyJuice-steered T2I models are significantly more effective at deceiving SIDs (up to 84%) compared to their unsteered counterparts. We also show that the steering directions can be estimated efficiently at lower resolutions and transferred to higher resolutions using simple interpolation, reducing computational overhead. Finally, tuning SID models on PolyJuice-augmented datasets notably enhances the performance of the detectors (up to 30%).