Leveraging Large-scale Multimedia Datasets to Refine Content Moderation Models
This work addresses the problem of efficient and scalable content moderation for digital platforms, offering an incremental improvement over existing methods by automating data annotation.
The paper tackles the challenge of training accurate content moderation models due to limited task-specific data and high annotation costs by proposing the CM-Refinery framework, which leverages large-scale multimedia datasets to automatically extend training data with hard examples, achieving accuracy improvements of 1.32% and 1.94% on NSFW and disturbing content detection tasks while reducing human annotation by up to 92.54%.
The sheer volume of online user-generated content has rendered content moderation technologies essential in order to protect digital platform audiences from content that may cause anxiety, worry, or concern. Despite the efforts towards developing automated solutions to tackle this problem, creating accurate models remains challenging due to the lack of adequate task-specific training data. The fact that manually annotating such data is a highly demanding procedure that could severely affect the annotators' emotional well-being is directly related to the latter limitation. In this paper, we propose the CM-Refinery framework that leverages large-scale multimedia datasets to automatically extend initial training datasets with hard examples that can refine content moderation models, while significantly reducing the involvement of human annotators. We apply our method on two model adaptation strategies designed with respect to the different challenges observed while collecting data, i.e. lack of (i) task-specific negative data or (ii) both positive and negative data. Additionally, we introduce a diversity criterion applied to the data collection process that further enhances the generalization performance of the refined models. The proposed method is evaluated on the Not Safe for Work (NSFW) and disturbing content detection tasks on benchmark datasets achieving 1.32% and 1.94% accuracy improvements compared to the state of the art, respectively. Finally, it significantly reduces human involvement, as 92.54% of data are automatically annotated in case of disturbing content while no human intervention is required for the NSFW task.