Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
This addresses the need for better training data for NLG models in hate speech countering, though it is incremental as it builds on existing resources.
The paper tackled the lack of multi-turn dialogue datasets for hate speech countering by presenting DIALOCONAN, a dataset with over 3000 fictitious multi-turn dialogues between a hater and an NGO operator, covering 6 hate targets.
Fighting online hate speech is a challenge that is usually addressed using Natural Language Processing via automatic detection and removal of hate content. Besides this approach, counter narratives have emerged as an effective tool employed by NGOs to respond to online hate on social media platforms. For this reason, Natural Language Generation is currently being studied as a way to automatize counter narrative writing. However, the existing resources necessary to train NLG models are limited to 2-turn interactions (a hate speech and a counter narrative as response), while in real life, interactions can consist of multiple turns. In this paper, we present a hybrid approach for dialogical data collection, which combines the intervention of human expert annotators over machine generated dialogues obtained using 19 different configurations. The result of this work is DIALOCONAN, the first dataset comprising over 3000 fictitious multi-turn dialogues between a hater and an NGO operator, covering 6 targets of hate.