SEER: Semantic Enhancement and Emotional Reasoning Network for Multimodal Fake News Detection
This work addresses the problem of detecting fake news in multimodal content for social media and information verification, representing an incremental improvement by integrating emotional features and semantic enhancement.
The paper tackles multimodal fake news detection by proposing the SEER network, which enhances semantic understanding using large multimodal models and incorporates emotional reasoning based on the tendency of fake news to contain negative emotions, achieving superior performance over state-of-the-art baselines on two real-world datasets.
Previous studies on multimodal fake news detection mainly focus on the alignment and integration of cross-modal features, as well as the application of text-image consistency. However, they overlook the semantic enhancement effects of large multimodal models and pay little attention to the emotional features of news. In addition, people find that fake news is more inclined to contain negative emotions than real ones. Therefore, we propose a novel Semantic Enhancement and Emotional Reasoning (SEER) Network for multimodal fake news detection. We generate summarized captions for image semantic understanding and utilize the products of large multimodal models for semantic enhancement. Inspired by the perceived relationship between news authenticity and emotional tendencies, we propose an expert emotional reasoning module that simulates real-life scenarios to optimize emotional features and infer the authenticity of news. Extensive experiments on two real-world datasets demonstrate the superiority of our SEER over state-of-the-art baselines.