Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning
This addresses the challenge of adapting quickly to new image manipulation techniques for applications in media forensics and security, though it is incremental as it builds on existing weakly-supervised learning frameworks.
The paper tackles the problem of detecting manipulated images without needing expensive pixel-level annotations, proposing a weakly-supervised method that uses only binary image-level labels and achieves competitive performance compared to fully-supervised approaches in both in-distribution and out-of-distribution evaluations.
As advanced image manipulation techniques emerge, detecting the manipulation becomes increasingly important. Despite the success of recent learning-based approaches for image manipulation detection, they typically require expensive pixel-level annotations to train, while exhibiting degraded performance when testing on images that are differently manipulated compared with training images. To address these limitations, we propose weakly-supervised image manipulation detection, such that only binary image-level labels (authentic or tampered with) are required for training purpose. Such a weakly-supervised setting can leverage more training images and has the potential to adapt quickly to new manipulation techniques. To improve the generalization ability, we propose weakly-supervised self-consistency learning (WSCL) to leverage the weakly annotated images. Specifically, two consistency properties are learned: multi-source consistency (MSC) and inter-patch consistency (IPC). MSC exploits different content-agnostic information and enables cross-source learning via an online pseudo label generation and refinement process. IPC performs global pair-wise patch-patch relationship reasoning to discover a complete region of manipulation. Extensive experiments validate that our WSCL, even though is weakly supervised, exhibits competitive performance compared with fully-supervised counterpart under both in-distribution and out-of-distribution evaluations, as well as reasonable manipulation localization ability.