Effectiveness of Counter-Speech against Abusive Content: A Multidimensional Annotation and Classification Study
This work addresses the challenge of evaluating counter-speech effectiveness for online platforms and researchers, though it is incremental in applying existing methods to a new annotation scheme.
The paper tackles the problem of assessing the effectiveness of counter-speech against online hate speech by proposing a computational framework with six dimensions, achieving strong classification results with average F1 scores of 0.94 and 0.96.
Counter-speech (CS) is a key strategy for mitigating online Hate Speech (HS), yet defining the criteria to assess its effectiveness remains an open challenge. We propose a novel computational framework for CS effectiveness classification, grounded in linguistics, communication and argumentation concepts. Our framework defines six core dimensions - Clarity, Evidence, Emotional Appeal, Rebuttal, Audience Adaptation, and Fairness - which we use to annotate 4,214 CS instances from two benchmark datasets, resulting in a novel linguistic resource released to the community. In addition, we propose two classification strategies, multi-task and dependency-based, achieving strong results (0.94 and 0.96 average F1 respectively on both expert- and user-written CS), outperforming standard baselines, and revealing strong interdependence among dimensions.