Event Stream based Multi-Modal Video Anomaly Detection: A Benchmark Dataset and Algorithms
This work addresses the need for robust video anomaly detection in real-world surveillance by leveraging event cameras, providing a benchmark and method that outperforms prior approaches.
The paper introduces EVAD, a framework combining conventional video and event camera streams for video anomaly detection, and constructs a large-scale benchmark dataset with 6.3 billion events and 376,368 frames. EVAD outperforms existing methods on multiple benchmarks, demonstrating improved robustness to challenging conditions like illumination variations and fast motion.
Video anomaly detection (VAD) is critical for automated surveillance but remains fragile under challenging conditions such as illumination variations, fast motion, and complex backgrounds when relying solely on visible light videos. To address these limitations, we propose EVAD, an event enhanced VAD framework that jointly exploits conventional video and event streams captured by bio inspired event cameras. Event sensors asynchronously capture brightness changes with high temporal resolution, offering robustness to motion blur and extreme lighting, and providing motion salient cues complementary to video based visual information. To support multi modal VAD research, we construct a large scale visible event benchmark comprising 6.3 billion events and 376,368 video frames collected under diverse illumination levels, motion patterns, and background complexities, filling the gap of realistic and scalable datasets for event based anomaly detection. Building upon this dataset, we design a contrastive multi modal pretraining framework to learn discriminative event representations by aligning semantic embeddings across event streams, visible videos, and textual descriptions. An adaptive fusion module then dynamically integrates event based temporal cues with video based spatial semantics, improving robustness to environmental disturbances. Experiments on benchmarks and the proposed TJUTCM Pha dataset demonstrate that E VAD consistently outperforms methods, validating the effectiveness of event-based sensing for VAD in real world scenarios.