ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information
This work addresses the challenge of harmful content detection for online environments, particularly in low-resource settings, but it is incremental as it builds on existing methods for data generation.
The paper tackles the problem of detecting harmful content in NLP by addressing data scarcity and inconsistent definitions, proposing the ToxiCraft framework to generate synthetic harmful information, which improves detection model robustness and adaptability, achieving results surpassing or close to gold labels.
In different NLP tasks, detecting harmful content is crucial for online environments, especially with the growing influence of social media. However, previous research has two main issues: 1) a lack of data in low-resource settings, and 2) inconsistent definitions and criteria for judging harmful content, requiring classification models to be robust to spurious features and diverse. We propose Toxicraft, a novel framework for synthesizing datasets of harmful information to address these weaknesses. With only a small amount of seed data, our framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information. Experimentation across various datasets showcases a notable enhancement in detection model robustness and adaptability, surpassing or close to the gold labels.