IP-Bench: Benchmark for Image Protection Methods in Image-to-Video Generation Scenarios
This work addresses the problem of misuse in I2V generation for security and content moderation, but it is incremental as it focuses on benchmarking rather than new protection methods.
The authors tackled the lack of a unified benchmark for evaluating image protection methods in image-to-video generation scenarios, proposing IP-Bench to systematically assess 6 protection methods and 5 I2V models, including robustness against attacks and transferability analysis.
With the rapid advancement of image-to-video (I2V) generation models, their potential for misuse in creating malicious content has become a significant concern. For instance, a single image can be exploited to generate a fake video, which can be used to attract attention and gain benefits. This phenomenon is referred to as an I2V generation misuse. Existing image protection methods suffer from the absence of a unified benchmark, leading to an incomplete evaluation framework. Furthermore, these methods have not been systematically assessed in I2V generation scenarios and against preprocessing attacks, which complicates the evaluation of their effectiveness in real-world deployment scenarios.To address this challenge, we propose IP-Bench (Image Protection Bench), the first systematic benchmark designed to evaluate protection methods in I2V generation scenarios. This benchmark examines 6 representative protection methods and 5 state-of-the-art I2V models. Furthermore, our work systematically evaluates protection methods' robustness with two robustness attack strategies under practical scenarios and analyzes their cross-model & cross-modality transferability. Overall, IP-Bench establishes a systematic, reproducible, and extensible evaluation framework for image protection methods in I2V generation scenarios.