Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitions
This work addresses the problem of inconsistent hate speech definitions for NLP researchers, offering a modular approach to improve dataset creation and annotation, but it is incremental as it builds on existing interdisciplinary insights without introducing new computational methods.
The paper tackles the subjectivity and inconsistency in defining hate speech for NLP tasks by proposing a modular framework of five criteria, developed with legal and social science perspectives, to help researchers create more precise definitions and annotation guidelines.
\textbf{Offensive Content Warning}: This paper contains offensive language only for providing examples that clarify this research and do not reflect the authors' opinions. Please be aware that these examples are offensive and may cause you distress. The subjectivity of recognizing \textit{hate speech} makes it a complex task. This is also reflected by different and incomplete definitions in NLP. We present \textit{hate speech} criteria, developed with perspectives from law and social science, with the aim of helping researchers create more precise definitions and annotation guidelines on five aspects: (1) target groups, (2) dominance, (3) perpetrator characteristics, (4) type of negative group reference, and the (5) type of potential consequences/effects. Definitions can be structured so that they cover a more broad or more narrow phenomenon. As such, conscious choices can be made on specifying criteria or leaving them open. We argue that the goal and exact task developers have in mind should determine how the scope of \textit{hate speech} is defined. We provide an overview of the properties of English datasets from \url{hatespeechdata.com} that may help select the most suitable dataset for a specific scenario.