MAD-Sherlock: Multi-Agent Debate for Visual Misinformation Detection
This addresses the challenge of detecting out-of-context misinformation for online platforms and users, offering a domain-agnostic tool with improved trust, though it builds incrementally on multi-agent and multimodal approaches.
The paper tackled the problem of detecting visual misinformation by pairing images with misleading text, introducing MAD-Sherlock, a multi-agent debate system that achieved state-of-the-art accuracy with improvements of 2%, 3%, and 5% on three benchmarks.
One of the most challenging forms of misinformation involves pairing images with misleading text to create false narratives. Existing AI-driven detection systems often require domain-specific finetuning, limiting generalizability, and offer little insight into their decisions, hindering trust and adoption. We introduce MAD-Sherlock, a multi-agent debate system for out-of-context misinformation detection. MAD-Sherlock frames detection as a multi-agent debate, reflecting the diverse and conflicting discourse found online. Multimodal agents collaborate to assess contextual consistency and retrieve external information to support cross-context reasoning. Our framework is domain- and time-agnostic, requiring no finetuning, yet achieves state-of-the-art accuracy with in-depth explanations. Evaluated on NewsCLIPpings, VERITE, and MMFakeBench, it outperforms prior methods by 2%, 3%, and 5%, respectively. Ablation and user studies show that the debate and resultant explanations significantly improve detection performance and improve trust for both experts and non-experts, positioning MAD-Sherlock as a robust tool for autonomous citizen intelligence.