Multimodal Fusion with BERT and Attention Mechanism for Fake News Detection
This addresses the problem of fake news spreading on social media, which is a serious societal concern, but the approach is incremental.
The paper tackled fake news detection by fusing multimodal features from text and images, achieving a 3.1% accuracy improvement over the state-of-the-art on a public Twitter dataset.
Fake news detection is an important task for increasing the credibility of information on the media since fake news is constantly spreading on social media every day and it is a very serious concern in our society. Fake news is usually created by manipulating images, texts, and videos. In this paper, we present a novel method for detecting fake news by fusing multimodal features derived from textual and visual data. Specifically, we used a pre-trained BERT model to learn text features and a VGG-19 model pre-trained on the ImageNet dataset to extract image features. We proposed a scale-dot product attention mechanism to capture the relationship between text features and visual features. Experimental results showed that our approach performs better than the current state-of-the-art method on a public Twitter dataset by 3.1% accuracy.