Learning Video Salient Object Detection Progressively from Unlabeled Videos
This work addresses the problem of reducing annotation costs for video salient object detection, making it more accessible for applications like video editing or surveillance, though it is incremental as it builds on existing image-based techniques.
The paper tackles video salient object detection without using any annotated video data by proposing a progressive framework that locates and segments objects sequentially, leveraging similarities with image-based methods and compensating for motion information with dynamic saliency. Experimental results on five benchmarks show the method is competitive with fully supervised approaches and outperforms state-of-the-art weakly and unsupervised methods.
Recent deep learning-based video salient object detection (VSOD) has achieved some breakthrough, but these methods rely on expensive annotated videos with pixel-wise annotations, weak annotations, or part of the pixel-wise annotations. In this paper, based on the similarities and the differences between VSOD and image salient object detection (SOD), we propose a novel VSOD method via a progressive framework that locates and segments salient objects in sequence without utilizing any video annotation. To use the knowledge learned in the SOD dataset for VSOD efficiently, we introduce dynamic saliency to compensate for the lack of motion information of SOD during the locating process but retain the same fine segmenting process. Specifically, an algorithm for generating spatiotemporal location labels, which consists of generating high-saliency location labels and tracking salient objects in adjacent frames, is proposed. Based on these location labels, a two-stream locating network that introduces an optical flow branch for video salient object locating is presented. Although our method does not require labeled video at all, the experimental results on five public benchmarks of DAVIS, FBMS, ViSal, VOS, and DAVSOD demonstrate that our proposed method is competitive with fully supervised methods and outperforms the state-of-the-art weakly and unsupervised methods.