Detecting Misinformation in Multimedia Content through Cross-Modal Entity Consistency: A Dual Learning Approach
This addresses the problem of combating misinformation in multimodal social media content, representing an incremental advance by focusing on video and improving entity consistency representation.
The paper tackled detecting misinformation in video content by leveraging cross-modal entity consistency, proposing a Multimedia Misinformation Detection (MultiMD) framework with a dual learning approach that outperforms state-of-the-art baseline models.
The landscape of social media content has evolved significantly, extending from text to multimodal formats. This evolution presents a significant challenge in combating misinformation. Previous research has primarily focused on single modalities or text-image combinations, leaving a gap in detecting multimodal misinformation. While the concept of entity consistency holds promise in detecting multimodal misinformation, simplifying the representation to a scalar value overlooks the inherent complexities of high-dimensional representations across different modalities. To address these limitations, we propose a Multimedia Misinformation Detection (MultiMD) framework for detecting misinformation from video content by leveraging cross-modal entity consistency. The proposed dual learning approach allows for not only enhancing misinformation detection performance but also improving representation learning of entity consistency across different modalities. Our results demonstrate that MultiMD outperforms state-of-the-art baseline models and underscore the importance of each modality in misinformation detection. Our research provides novel methodological and technical insights into multimodal misinformation detection.