The Hidden Language of Harm: Examining the Role of Emojis in Harmful Online Communication and Content Moderation
This addresses the problem of improving content moderation on social media platforms for users and moderators, but it is incremental as it builds on existing text-based methods by focusing on emojis.
The study tackled the problem of emojis contributing to harmful online communication by analyzing their role in offensive Twitter messages and proposing an LLM-powered moderation pipeline that selectively replaces harmful emojis. The result showed that human evaluations confirmed the approach effectively reduced perceived offensiveness without sacrificing meaning, though no concrete numbers were provided.
Social media platforms have become central to modern communication, yet they also harbor offensive content that challenges platform safety and inclusivity. While prior research has primarily focused on textual indicators of offense, the role of emojis, ubiquitous visual elements in online discourse, remains underexplored. Emojis, despite being rarely offensive in isolation, can acquire harmful meanings through symbolic associations, sarcasm, and contextual misuse. In this work, we systematically examine emoji contributions to offensive Twitter messages, analyzing their distribution across offense categories and how users exploit emoji ambiguity. To address this, we propose an LLM-powered, multi-step moderation pipeline that selectively replaces harmful emojis while preserving the tweet's semantic intent. Human evaluations confirm our approach effectively reduces perceived offensiveness without sacrificing meaning. Our analysis also reveals heterogeneous effects across offense types, offering nuanced insights for online communication and emoji moderation.