Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidance
This work addresses video anomaly detection for security and surveillance applications, presenting an incremental improvement by focusing on under-explored text modality integration.
The paper tackles the problem of weakly supervised multimodal video anomaly detection by leveraging text guidance to enhance anomaly characterization and reduce false alarms, achieving state-of-the-art performance on UCF-Crime and XD-Violence datasets.
Weakly supervised multimodal video anomaly detection has gained significant attention, yet the potential of the text modality remains under-explored. Text provides explicit semantic information that can enhance anomaly characterization and reduce false alarms. However, extracting effective text features is challenging due to the inability of general-purpose language models to capture anomaly-specific nuances and the scarcity of relevant descriptions. Furthermore, multimodal fusion often suffers from redundancy and imbalance. To address these issues, we propose a novel text-guided framework. First, we introduce an in-context learning-based multi-stage text augmentation mechanism to generate high-quality anomaly text samples for fine-tuning the text feature extractor. Second, we design a multi-scale bottleneck Transformer fusion module that uses compressed bottleneck tokens to progressively integrate information across modalities, mitigating redundancy and imbalance. Experiments on UCF-Crime and XD-Violence demonstrate state-of-the-art performance.