Classification of Multimodal Hate Speech -- The Winning Solution of Hateful Memes Challenge
This work provides a strong solution for detecting hate speech in multimodal memes, which is a challenging problem for social media platforms.
The paper addresses the Hateful Memes challenge, a multimodal classification task for detecting hate speech in memes, where existing methods perform poorly. The proposed model, combining multimodal techniques with rules extracted from the training set, achieved a leading accuracy of 86.8% and an AUROC of 0.923.
Hateful Memes is a new challenge set for multimodal classification, focusing on detecting hate speech in multimodal memes. Difficult examples are added to the dataset to make it hard to rely on unimodal signals, which means only multimodal models can succeed. According to Kiela,the state-of-the-art methods perform poorly compared to humans (64.73% vs. 84.7% accuracy) on Hateful Memes. I propose a new model that combined multimodal with rules, which achieve the first ranking of accuracy and AUROC of 86.8% and 0.923 respectively. These rules are extracted from training set, and focus on improving the classification accuracy of difficult samples.