SceneGraMMi: Scene Graph-boosted Hybrid-fusion for Multi-Modal Misinformation Veracity Prediction
This addresses misinformation detection for social media and news platforms, representing an incremental improvement over existing multi-modal methods.
The paper tackles multi-modal misinformation detection by proposing SceneGraMMi, a scene graph-boosted hybrid-fusion approach that integrates scene graphs across modalities to capture semantic cues and cross-modal similarities, achieving state-of-the-art performance across four benchmark datasets.
Misinformation undermines individual knowledge and affects broader societal narratives. Despite growing interest in the research community in multi-modal misinformation detection, existing methods exhibit limitations in capturing semantic cues, key regions, and cross-modal similarities within multi-modal datasets. We propose SceneGraMMi, a Scene Graph-boosted Hybrid-fusion approach for Multi-modal Misinformation veracity prediction, which integrates scene graphs across different modalities to improve detection performance. Experimental results across four benchmark datasets show that SceneGraMMi consistently outperforms state-of-the-art methods. In a comprehensive ablation study, we highlight the contribution of each component, while Shapley values are employed to examine the explainability of the model's decision-making process.