Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization
This addresses the problem of segmenting tampered regions in images for digital forensics, representing an incremental improvement over existing deep learning methods.
The paper tackles image forgery localization by proposing a Multi-view Pixel-wise Contrastive algorithm (MPC) that pre-trains with contrastive loss to model pixel relationships, achieving higher generalization and robustness against post-processing compared to state-of-the-art methods.
Image forgery localization, which aims to segment tampered regions in an image, is a fundamental yet challenging digital forensic task. While some deep learning-based forensic methods have achieved impressive results, they directly learn pixel-to-label mappings without fully exploiting the relationship between pixels in the feature space. To address such deficiency, we propose a Multi-view Pixel-wise Contrastive algorithm (MPC) for image forgery localization. Specifically, we first pre-train the backbone network with the supervised contrastive loss to model pixel relationships from the perspectives of within-image, cross-scale and cross-modality. That is aimed at increasing intra-class compactness and inter-class separability. Then the localization head is fine-tuned using the cross-entropy loss, resulting in a better pixel localizer. The MPC is trained on three different scale training datasets to make a comprehensive and fair comparison with existing image forgery localization algorithms. Extensive experiments on the small, medium and large scale training datasets show that the proposed MPC achieves higher generalization performance and robustness against post-processing than the state-of-the-arts. Code will be available at https://github.com/multimediaFor/MPC.