Fight Fire with Fire: Fine-tuning Hate Detectors using Large Samples of Generated Hate Speech
This work addresses the data bottleneck in hate speech detection for social media and content moderation, offering a method to enhance model performance with synthetic data, though it is incremental as it builds on existing fine-tuning techniques.
The paper tackled the problem of poor generalization in hate speech detection due to scarce labeled data by using GPT to generate synthetic hate speech sequences for fine-tuning pretrained language models, resulting in significant and consistent improvements in generalization within and across data distributions.
Automatic hate speech detection is hampered by the scarcity of labeled datasetd, leading to poor generalization. We employ pretrained language models (LMs) to alleviate this data bottleneck. We utilize the GPT LM for generating large amounts of synthetic hate speech sequences from available labeled examples, and leverage the generated data in fine-tuning large pretrained LMs on hate detection. An empirical study using the models of BERT, RoBERTa and ALBERT, shows that this approach improves generalization significantly and consistently within and across data distributions. In fact, we find that generating relevant labeled hate speech sequences is preferable to using out-of-domain, and sometimes also within-domain, human-labeled examples.