WAVES: Benchmarking the Robustness of Image Watermarks
This addresses the need for reliable watermark evaluation in generative AI for content provenance, though it is incremental as it builds on existing benchmarking methods.
The authors tackled the problem of evaluating image watermark robustness by introducing WAVES, a benchmark that integrates detection and identification tasks with standardized stress tests, revealing vulnerabilities in modern algorithms.
In the burgeoning age of generative AI, watermarks act as identifiers of provenance and artificial content. We present WAVES (Watermark Analysis Via Enhanced Stress-testing), a benchmark for assessing image watermark robustness, overcoming the limitations of current evaluation methods. WAVES integrates detection and identification tasks and establishes a standardized evaluation protocol comprised of a diverse range of stress tests. The attacks in WAVES range from traditional image distortions to advanced, novel variations of diffusive, and adversarial attacks. Our evaluation examines two pivotal dimensions: the degree of image quality degradation and the efficacy of watermark detection after attacks. Our novel, comprehensive evaluation reveals previously undetected vulnerabilities of several modern watermarking algorithms. We envision WAVES as a toolkit for the future development of robust watermarks. The project is available at https://wavesbench.github.io/