On the Robustness of Watermarking for Autoregressive Image Generation
This work highlights critical security flaws in watermarking for AR image generation, undermining its reliability for detecting synthetic content and filtering training data, which is important for preventing misinformation and model collapse.
The paper studies watermarking for autoregressive image generation and shows that existing schemes are vulnerable to removal and forgery attacks, even with only a single watermarked reference image and no access to model parameters or secrets. The attacks enable watermark mimicry, where authentic images can be falsely detected as synthetic.
The proliferation of autoregressive (AR) image generators demands reliable detection and attribution of their outputs to mitigate misinformation, and to filter synthetic images from training data to prevent model collapse. To address this need, watermarking techniques, specifically designed for AR models, embed a subtle signal at generation time, enabling downstream verification through a corresponding watermark detector. In this work, we study these schemes and demonstrate their vulnerability to both watermark removal and forgery attacks. We assess existing attacks and further introduce three new attacks: (i) a vector-quantized regeneration removal attack, (ii) adversarial optimization-based attack, and (iii) a frequency injection attack. Our evaluation reveals that removal and forgery attacks can be effective with access to a single watermarked reference image and without access to original model parameters or watermarking secrets. Our findings indicate that existing watermarking schemes for AR image generation do not reliably support synthetic content detection for dataset filtering. Moreover, they enable Watermark Mimicry, whereby authentic images can be manipulated to imitate a generator's watermark and trigger false detection to prevent their inclusion in future model training.