Video Event Restoration Based on Keyframes for Video Anomaly Detection
This work addresses the problem of improving video anomaly detection for computer vision applications by proposing a novel paradigm, though it appears incremental as it builds on existing deep learning and transformer-based approaches.
The paper tackles video anomaly detection by introducing a new task of video event restoration based on keyframes, where a U-shaped Swin Transformer Network with Dual Skip Connections (USTN-DSC) is proposed and shown to outperform most existing methods on benchmarks.
Video anomaly detection (VAD) is a significant computer vision problem. Existing deep neural network (DNN) based VAD methods mostly follow the route of frame reconstruction or frame prediction. However, the lack of mining and learning of higher-level visual features and temporal context relationships in videos limits the further performance of these two approaches. Inspired by video codec theory, we introduce a brand-new VAD paradigm to break through these limitations: First, we propose a new task of video event restoration based on keyframes. Encouraging DNN to infer missing multiple frames based on video keyframes so as to restore a video event, which can more effectively motivate DNN to mine and learn potential higher-level visual features and comprehensive temporal context relationships in the video. To this end, we propose a novel U-shaped Swin Transformer Network with Dual Skip Connections (USTN-DSC) for video event restoration, where a cross-attention and a temporal upsampling residual skip connection are introduced to further assist in restoring complex static and dynamic motion object features in the video. In addition, we propose a simple and effective adjacent frame difference loss to constrain the motion consistency of the video sequence. Extensive experiments on benchmarks demonstrate that USTN-DSC outperforms most existing methods, validating the effectiveness of our method.