Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution
This work addresses hateful content detection for social media platforms, but it is incremental as it applies standard data augmentation to existing models.
The paper tackled hateful meme detection using multimodal models and data augmentation, achieving a 2% AUROC boost and final scores of 0.7439 AUROC and 0.7037 accuracy on the test set.
Hateful content detection is one of the areas where deep learning can and should make a significant difference. The Hateful Memes Challenge from Facebook helps fulfill such potential by challenging the contestants to detect hateful speech in multi-modal memes using deep learning algorithms. In this paper, we utilize multi-modal, pre-trained models VilBERT and Visual BERT. We improved models' performance by adding training datasets generated from data augmentation. Enlarging the training data set helped us get a more than 2% boost in terms of AUROC with the Visual BERT model. Our approach achieved 0.7439 AUROC along with an accuracy of 0.7037 on the challenge's test set, which revealed remarkable progress.