Labeling Comic Mischief Content in Online Videos with a Multimodal Hierarchical-Cross-Attention Model
This work addresses the problem of identifying subtle questionable content for online users, but it is incremental as it builds on existing multimodal detection methods.
The paper tackles detecting comic mischief content in online videos, which combines humor with elements like violence or adult content, by proposing a multimodal hierarchical-cross-attention model (HICCAP) and releasing a new dataset; the results show significant improvements over baselines and state-of-the-art models in detection and classification tasks.
We address the challenge of detecting questionable content in online media, specifically the subcategory of comic mischief. This type of content combines elements such as violence, adult content, or sarcasm with humor, making it difficult to detect. Employing a multimodal approach is vital to capture the subtle details inherent in comic mischief content. To tackle this problem, we propose a novel end-to-end multimodal system for the task of comic mischief detection. As part of this contribution, we release a novel dataset for the targeted task consisting of three modalities: video, text (video captions and subtitles), and audio. We also design a HIerarchical Cross-attention model with CAPtions (HICCAP) to capture the intricate relationships among these modalities. The results show that the proposed approach makes a significant improvement over robust baselines and state-of-the-art models for comic mischief detection and its type classification. This emphasizes the potential of our system to empower users, to make informed decisions about the online content they choose to see. In addition, we conduct experiments on the UCF101, HMDB51, and XD-Violence datasets, comparing our model against other state-of-the-art approaches showcasing the outstanding performance of our proposed model in various scenarios.