Parsimonious Argument Annotations for Hate Speech Counter-narratives
This work addresses the challenge of creating effective automated counter-narratives for hate speech, but it is incremental as it builds on existing datasets and methods.
The researchers tackled the problem of generating counter-narratives for hate speech by enriching the Hateval corpus with argumentative annotations, but preliminary results show that automatic detection of these annotations varies, with some aspects performing close to human levels while others achieve only low to moderate agreement.
We present an enrichment of the Hateval corpus of hate speech tweets (Basile et. al 2019) aimed to facilitate automated counter-narrative generation. Comparably to previous work (Chung et. al. 2019), manually written counter-narratives are associated to tweets. However, this information alone seems insufficient to obtain satisfactory language models for counter-narrative generation. That is why we have also annotated tweets with argumentative information based on Wagemanns (2016), that we believe can help in building convincing and effective counter-narratives for hate speech against particular groups. We discuss adequacies and difficulties of this annotation process and present several baselines for automatic detection of the annotated elements. Preliminary results show that automatic annotators perform close to human annotators to detect some aspects of argumentation, while others only reach low or moderate level of inter-annotator agreement.