EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM
This addresses a critical challenge in digital forensics and media credibility by improving detection of advanced image forgeries, though it is incremental as it builds on existing LLM capabilities for a specific bottleneck.
The paper tackles the problem of localizing forged regions in images edited by diffusion-based techniques, which current forensic methods struggle with, and presents a framework integrating a multimodal LLM that outperforms previous approaches on datasets like MagicBrush, AutoSplice, and PerfBrush, achieving higher mIoU and F1-scores.
Image editing technologies are tools used to transform, adjust, remove, or otherwise alter images. Recent research has significantly improved the capabilities of image editing tools, enabling the creation of photorealistic and semantically informed forged regions that are nearly indistinguishable from authentic imagery, presenting new challenges in digital forensics and media credibility. While current image forensic techniques are adept at localizing forged regions produced by traditional image manipulation methods, current capabilities struggle to localize regions created by diffusion-based techniques. To bridge this gap, we present a novel framework that integrates a multimodal Large Language Model (LLM) for enhanced reasoning capabilities to localize tampered regions in images produced by diffusion model-based editing methods. By leveraging the contextual and semantic strengths of LLMs, our framework achieves promising results on MagicBrush, AutoSplice, and PerfBrush (novel diffusion-based dataset) datasets, outperforming previous approaches in mIoU and F1-score metrics. Notably, our method excels on the PerfBrush dataset, a self-constructed test set featuring previously unseen types of edits. Here, where traditional methods typically falter, achieving markedly low scores, our approach demonstrates promising performance.