Temporal-Spatial Feature Pyramid for Video Saliency Detection
This work addresses video saliency detection for applications like video analysis and compression, but it appears incremental as it builds on existing feature pyramid and encoder-decoder approaches.
The paper tackled video saliency detection by proposing a 3D fully convolutional encoder-decoder architecture that combines multi-level temporal-spatial features, resulting in improved accuracy and real-time performance, with experimental results showing it outperforms state-of-the-art methods on multiple benchmarks.
Multi-level features are important for saliency detection. Better combination and use of multi-level features with time information can greatly improve the accuracy of the video saliency model. In order to fully combine multi-level features and make it serve the video saliency model, we propose a 3D fully convolutional encoder-decoder architecture for video saliency detection, which combines scale, space and time information for video saliency modeling. The encoder extracts multi-scale temporal-spatial features from the input continuous video frames, and then constructs temporal-spatial feature pyramid through temporal-spatial convolution and top-down feature integration. The decoder performs hierarchical decoding of temporal-spatial features from different scales, and finally produces a saliency map from the integration of multiple video frames. Our model is simple yet effective, and can run in real time. We perform abundant experiments, and the results indicate that the well-designed structure can improve the precision of video saliency detection significantly. Experimental results on three purely visual video saliency benchmarks and six audio-video saliency benchmarks demonstrate that our method outperforms the existing state-of-the-art methods.