NewsRECON: News article REtrieval for image CONtextualization
This addresses a practical challenge for journalists and forensic experts in verifying news credibility and combating misinformation, though it is an incremental improvement over existing methods.
The paper tackles the problem of identifying when and where news images were taken when reverse image search fails, by linking images to relevant news articles using metadata, achieving new state-of-the-art results on benchmarks like TARA and 5Pils-OOC.
Identifying when and where a news image was taken is crucial for journalists and forensic experts to produce credible stories and debunk misinformation. While many existing methods rely on reverse image search (RIS) engines, these tools often fail to return results, thereby limiting their practical applicability. In this work, we address the challenging scenario where RIS evidence is unavailable. We introduce NewsRECON, a method that links images to relevant news articles to infer their date and location from article metadata. NewsRECON leverages a corpus of over 90,000 articles and integrates: (1) a bi-encoder for retrieving event-relevant articles; (2) two cross-encoders for reranking articles by location and event consistency. Experiments on the TARA and 5Pils-OOC show that NewsRECON outperforms prior work and can be combined with a multimodal large language model to achieve new SOTA results in the absence of RIS evidence. We make our code available.