Detecting Community Sensitive Norm Violations in Online Conversations
This addresses the need for more comprehensive moderation tools for online communities, though it is incremental by extending existing toxicity detection to include other norm violations.
The paper tackles the problem of detecting a broader spectrum of community norm violations beyond just toxicity in online conversations, by introducing a new dataset and models that achieve high performance in context- and community-sensitive detection.
Online platforms and communities establish their own norms that govern what behavior is acceptable within the community. Substantial effort in NLP has focused on identifying unacceptable behaviors and, recently, on forecasting them before they occur. However, these efforts have largely focused on toxicity as the sole form of community norm violation. Such focus has overlooked the much larger set of rules that moderators enforce. Here, we introduce a new dataset focusing on a more complete spectrum of community norms and their violations in the local conversational and global community contexts. We introduce a series of models that use this data to develop context- and community-sensitive norm violation detection, showing that these changes give high performance.