Malicious or Benign? Towards Effective Content Moderation for Children's Videos
This addresses the issue of protecting young children from harmful video content on online platforms, though it appears incremental as it builds upon existing video classification methods.
The paper tackles the problem of detecting malicious content in children's videos, which often evades current moderation tools, by introducing a toolkit that includes an annotation tool, a dataset with difficult test cases, and a benchmark suite of state-of-the-art video classification models.
Online video platforms receive hundreds of hours of uploads every minute, making manual content moderation impossible. Unfortunately, the most vulnerable consumers of malicious video content are children from ages 1-5 whose attention is easily captured by bursts of color and sound. Scammers attempting to monetize their content may craft malicious children's videos that are superficially similar to educational videos, but include scary and disgusting characters, violent motions, loud music, and disturbing noises. Prominent video hosting platforms like YouTube have taken measures to mitigate malicious content on their platform, but these videos often go undetected by current content moderation tools that are focused on removing pornographic or copyrighted content. This paper introduces our toolkit Malicious or Benign for promoting research on automated content moderation of children's videos. We present 1) a customizable annotation tool for videos, 2) a new dataset with difficult to detect test cases of malicious content and 3) a benchmark suite of state-of-the-art video classification models.