Multimodal Claim Extraction for Fact-Checking
This addresses the challenge of misinformation in multimodal social media for fact-checkers, but it is incremental as it builds on existing multimodal methods.
The paper tackled the problem of extracting claims from multimodal social media posts for automated fact-checking, introducing the first benchmark and showing that baseline multimodal LLMs struggle, while their proposed MICE framework improved performance in intent-critical cases.
Automated Fact-Checking (AFC) relies on claim extraction as a first step, yet existing methods largely overlook the multimodal nature of today's misinformation. Social media posts often combine short, informal text with images such as memes, screenshots, and photos, creating challenges that differ from both text-only claim extraction and well-studied multimodal tasks like image captioning or visual question answering. In this work, we present the first benchmark for multimodal claim extraction from social media, consisting of posts containing text and one or more images, annotated with gold-standard claims derived from real-world fact-checkers. We evaluate state-of-the-art multimodal LLMs (MLLMs) under a three-part evaluation framework (semantic alignment, faithfulness, and decontextualization) and find that baseline MLLMs struggle to model rhetorical intent and contextual cues. To address this, we introduce MICE, an intent-aware framework which shows improvements in intent-critical cases.