Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations
This addresses the problem of unreliable content moderation in online harm detection, particularly for small corpora, and is incremental as it builds on prior toxicity work.
The study tackled the vulnerability of cyberbullying classifiers to model-agnostic lexical perturbations, showing that such substitutions significantly reduce classifier performance, and using perturbed samples for augmentation improves robustness with a slight trade-off in overall task performance.
A limited amount of studies investigates the role of model-agnostic adversarial behavior in toxic content classification. As toxicity classifiers predominantly rely on lexical cues, (deliberately) creative and evolving language-use can be detrimental to the utility of current corpora and state-of-the-art models when they are deployed for content moderation. The less training data is available, the more vulnerable models might become. This study is, to our knowledge, the first to investigate the effect of adversarial behavior and augmentation for cyberbullying detection. We demonstrate that model-agnostic lexical substitutions significantly hurt classifier performance. Moreover, when these perturbed samples are used for augmentation, we show models become robust against word-level perturbations at a slight trade-off in overall task performance. Augmentations proposed in prior work on toxicity prove to be less effective. Our results underline the need for such evaluations in online harm areas with small corpora. The perturbed data, models, and code are available for reproduction at https://github.com/cmry/augtox