Can GPT tell us why these images are synthesized? Empowering Multimodal Large Language Models for Forensics
This addresses the challenge of combating AI-generated content for security and forensics applications, though it is incremental as it adapts existing LLMs rather than introducing a fundamentally new approach.
The paper tackles the problem of detecting AI-generated or manipulated images by applying multimodal large language models (LLMs) to forgery detection, achieving accuracies of 92.1% on Autosplice and 86.3% on LaMa datasets, which are competitive with state-of-the-art methods.
The rapid development of generative AI facilitates content creation and makes image manipulation easier and more difficult to detect. While multimodal Large Language Models (LLMs) have encoded rich world knowledge, they are not inherently tailored for combating AI-generated Content (AIGC) and struggle to comprehend local forgery details. In this work, we investigate the application of multimodal LLMs in forgery detection. We propose a framework capable of evaluating image authenticity, localizing tampered regions, providing evidence, and tracing generation methods based on semantic tampering clues. Our method demonstrates that the potential of LLMs in forgery analysis can be effectively unlocked through meticulous prompt engineering and the application of few-shot learning techniques. We conduct qualitative and quantitative experiments and show that GPT4V can achieve an accuracy of 92.1% in Autosplice and 86.3% in LaMa, which is competitive with state-of-the-art AIGC detection methods. We further discuss the limitations of multimodal LLMs in such tasks and propose potential improvements.