STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation
This work addresses the problem of efficient and accurate video instance segmentation for computer vision applications, offering a simpler alternative to complex pipelines, though it is incremental as it builds on existing instance segmentation methods.
The paper tackles video instance segmentation by proposing a simple single-stage framework with a tracking head, using bi-directional spatio-temporal contrastive learning and instance-wise temporal consistency to improve instance association accuracy, achieving competitive results on datasets like YouTube-VIS-2019, YouTube-VIS-2021, and OVIS-2021.
Video Instance Segmentation (VIS) is a task that simultaneously requires classification, segmentation, and instance association in a video. Recent VIS approaches rely on sophisticated pipelines to achieve this goal, including RoI-related operations or 3D convolutions. In contrast, we present a simple and efficient single-stage VIS framework based on the instance segmentation method CondInst by adding an extra tracking head. To improve instance association accuracy, a novel bi-directional spatio-temporal contrastive learning strategy for tracking embedding across frames is proposed. Moreover, an instance-wise temporal consistency scheme is utilized to produce temporally coherent results. Experiments conducted on the YouTube-VIS-2019, YouTube-VIS-2021, and OVIS-2021 datasets validate the effectiveness and efficiency of the proposed method. We hope the proposed framework can serve as a simple and strong alternative for many other instance-level video association tasks.