Detecting Hateful Memes Using a Multimodal Deep Ensemble
This work provides an incremental improvement in the detection of hateful memes for social media platforms and content moderators.
This paper addresses the challenge of detecting hateful memes, which combine visual and linguistic elements. The authors propose improvements to recent visual-linguistic Transformer architectures, resulting in a model that significantly outperforms baselines and achieved 5th place out of over 3,100 participants on a leaderboard.
While significant progress has been made using machine learning algorithms to detect hate speech, important technical challenges still remain to be solved in order to bring their performance closer to human accuracy. We investigate several of the most recent visual-linguistic Transformer architectures and propose improvements to increase their performance for this task. The proposed model outperforms the baselines by a large margin and ranks 5$^{th}$ on the leaderboard out of 3,100+ participants.