Learning from Mistakes: Enhancing Harmful Meme Detection via Misjudgment Risk Patterns
This addresses the challenge of harmful meme detection for online content moderation, offering a novel method to reduce misjudgments, though it is an incremental improvement over existing MLLM-based techniques.
The paper tackles the problem of detecting harmful memes with implicit expressions like irony and metaphor, where existing methods often misjudge, by introducing PatMD, which learns from misjudgment risk patterns to guide multimodal large language models (MLLMs). The result is an average improvement of 8.30% in F1-score and 7.71% in accuracy across 5 harmful detection tasks on a benchmark of 6,626 memes.
Internet memes have emerged as a popular multimodal medium, yet they are increasingly weaponized to convey harmful opinions through subtle rhetorical devices like irony and metaphor. Existing detection approaches, including MLLM-based techniques, struggle with these implicit expressions, leading to frequent misjudgments. This paper introduces PatMD, a novel approach that improves harmful meme detection by learning from and proactively mitigating these potential misjudgment risks. Our core idea is to move beyond superficial content-level matching and instead identify the underlying misjudgment risk patterns, proactively guiding the MLLMs to avoid known misjudgment pitfalls. We first construct a knowledge base where each meme is deconstructed into a misjudgment risk pattern explaining why it might be misjudged, either overlooking harmful undertones (false negative) or overinterpreting benign content (false positive). For a given target meme, PatMD retrieves relevant patterns and utilizes them to dynamically guide the MLLM's reasoning. Experiments on a benchmark of 6,626 memes across 5 harmful detection tasks show that PatMD outperforms state-of-the-art baselines, achieving an average of 8.30\% improvement in F1-score and 7.71\% improvement in accuracy, demonstrating strong generalizability and improved detection capability of harmful memes.