A Holistic Approach to Undesired Content Detection in the Real World
This addresses content moderation for platforms needing robust detection of harmful content, but it appears incremental as it combines existing techniques into a comprehensive pipeline.
The paper tackles the problem of real-world content moderation by developing a holistic natural language classification system that detects multiple categories of undesired content, resulting in high-quality classifiers that outperform off-the-shelf models.
We present a holistic approach to building a robust and useful natural language classification system for real-world content moderation. The success of such a system relies on a chain of carefully designed and executed steps, including the design of content taxonomies and labeling instructions, data quality control, an active learning pipeline to capture rare events, and a variety of methods to make the model robust and to avoid overfitting. Our moderation system is trained to detect a broad set of categories of undesired content, including sexual content, hateful content, violence, self-harm, and harassment. This approach generalizes to a wide range of different content taxonomies and can be used to create high-quality content classifiers that outperform off-the-shelf models.