PAMI-AD: An Activity Detector Exploiting Part-attention and Motion Information in Surveillance Videos
It addresses the problem of detecting activities in surveillance videos for security and monitoring applications, representing an incremental improvement over existing methods.
The paper tackles activity detection in surveillance videos by proposing a system with four modules, including a part-attention mechanism for person-centered activities and a localization masking method for vehicle-centered activities, achieving state-of-the-art results on the VIRAT dataset and winning first place in the TRECVID 2021 ActEV challenge.
Activity detection in surveillance videos is a challenging task caused by small objects, complex activity categories, its untrimmed nature, etc. Existing methods are generally limited in performance due to inaccurate proposals, poor classifiers or inadequate post-processing method. In this work, we propose a comprehensive and effective activity detection system in untrimmed surveillance videos for person-centered and vehicle-centered activities. It consists of four modules, i.e., object localizer, proposal filter, activity classifier and activity refiner. For person-centered activities, a novel part-attention mechanism is proposed to explore detailed features in different body parts. As for vehicle-centered activities, we propose a localization masking method to jointly encode motion and foreground attention features. We conduct experiments on the large-scale activity detection datasets VIRAT, and achieve the best results for both groups of activities. Furthermore, our team won the 1st place in the TRECVID 2021 ActEV challenge.