Evidence Triangulation for Multimodal Fact-Checking in the Wild
For researchers in multimodal fact-checking, this work provides a more realistic benchmark and a novel model that improves detection of real-world misinformation.
The paper introduces X-POSE, a real-world benchmark for multimodal fact-checking, and TRENT, a model that uses evidence triangulation via three parallel cross-attention streams and relational fusion. TRENT outperforms state-of-the-art models and commercial VLMs on this benchmark.
The proliferation of multimedia content on social platforms has fueled multimodal misinformation, where images are used to reinforce false claims. Consequently, Multimodal Fact-Checking (MFC) has emerged as an increasingly important research area. However, current progress is hindered by a reliance on synthetic training data and curated benchmarks that fail to capture the complexity of in-the-wild data. Furthermore, existing detection models rely on restricted intra-modality consistency or unconstrained all-to-all fusion, failing to capture nuanced relations between posts and external evidence. To address these limitations, we introduce X-POSE, a benchmark of real-world, community-annotated multimodal posts from X (formerly Twitter), augmented with full-length news articles retrieved via VLM-optimized search. Additionally, we propose TRENT, a novel MFC model that performs evidence triangulation using three parallel cross-attention streams alongside a relational fusion mechanism that explicitly models entailment and contradiction. Extensive evaluations demonstrate that TRENT consistently outperforms state-of-the-art specialized models and commercial VLMs. The code, prompt templates, and dataset are available at https://github.com/stevejpapad/evidence-triangulation