Multi-Modal Video Forensic Platform for Investigating Post-Terrorist Attack Scenarios
This addresses a critical problem for law enforcement agencies in forensic investigations, though it is incremental as it extends existing video analytic platforms by adding audio capabilities.
The paper tackles the challenge of analyzing thousands of hours of video footage after terrorist attacks by developing a multi-modal platform that integrates visual and audio analytics, enabling faster investigation through indexing and similarity search.
The forensic investigation of a terrorist attack poses a significant challenge to the investigative authorities, as often several thousand hours of video footage must be viewed. Large scale Video Analytic Platforms (VAP) assist law enforcement agencies (LEA) in identifying suspects and securing evidence. Current platforms focus primarily on the integration of different computer vision methods and thus are restricted to a single modality. We present a video analytic platform that integrates visual and audio analytic modules and fuses information from surveillance cameras and video uploads from eyewitnesses. Videos are analyzed according their acoustic and visual content. Specifically, Audio Event Detection is applied to index the content according to attack-specific acoustic concepts. Audio similarity search is utilized to identify similar video sequences recorded from different perspectives. Visual object detection and tracking are used to index the content according to relevant concepts. Innovative user-interface concepts are introduced to harness the full potential of the heterogeneous results of the analytical modules, allowing investigators to more quickly follow-up on leads and eyewitness reports.