Detecting Out-of-Context Image-Caption Pairs in News: A Counter-Intuitive Method
This addresses misinformation detection in news and social media, but is incremental as it focuses on dataset generation and preliminary analysis.
The paper tackles the problem of detecting out-of-context image-caption pairs in news to combat misinformation, by generating 6800 images using DALL-E 2 and Stable-Diffusion for dataset creation and evaluating image similarity methods.
The growth of misinformation and re-contextualized media in social media and news leads to an increasing need for fact-checking methods. Concurrently, the advancement in generative models makes cheapfakes and deepfakes both easier to make and harder to detect. In this paper, we present a novel approach using generative image models to our advantage for detecting Out-of-Context (OOC) use of images-caption pairs in news. We present two new datasets with a total of $6800$ images generated using two different generative models including (1) DALL-E 2, and (2) Stable-Diffusion. We are confident that the method proposed in this paper can further research on generative models in the field of cheapfake detection, and that the resulting datasets can be used to train and evaluate new models aimed at detecting cheapfakes. We run a preliminary qualitative and quantitative analysis to evaluate the performance of each image generation model for this task, and evaluate a handful of methods for computing image similarity.