Placing M-Phasis on the Plurality of Hate: A Feature-Based Corpus of Hate Online
This addresses the problem of limited real-life applicability in hate speech classifiers for researchers and practitioners by providing a more nuanced dataset, though it is incremental as it builds on existing corpus creation efforts.
The authors tackled the oversimplification of hate speech detection by creating the M-Phasis corpus, a dataset of ~9k German and French comments annotated with 23 features to capture complex hate types, achieving high inter-annotator agreements (0.77 <= k <= 1).
Even though hate speech (HS) online has been an important object of research in the last decade, most HS-related corpora over-simplify the phenomenon of hate by attempting to label user comments as "hate" or "neutral". This ignores the complex and subjective nature of HS, which limits the real-life applicability of classifiers trained on these corpora. In this study, we present the M-Phasis corpus, a corpus of ~9k German and French user comments collected from migration-related news articles. It goes beyond the "hate"-"neutral" dichotomy and is instead annotated with 23 features, which in combination become descriptors of various types of speech, ranging from critical comments to implicit and explicit expressions of hate. The annotations are performed by 4 native speakers per language and achieve high (0.77 <= k <= 1) inter-annotator agreements. Besides describing the corpus creation and presenting insights from a content, error and domain analysis, we explore its data characteristics by training several classification baselines.