Learning to Track Instances without Video Annotations
This reduces annotation costs for video instance tracking, making it more accessible for applications like surveillance or autonomous driving, though it is incremental as it builds on existing instance segmentation methods.
The paper tackles the problem of tracking multiple instances in videos without requiring frame-wise annotations by introducing a semi-supervised framework that uses only labeled images and unlabeled videos, achieving comparable or better performance than fully-supervised methods on YouTube-VIS and PoseTrack datasets.
Tracking segmentation masks of multiple instances has been intensively studied, but still faces two fundamental challenges: 1) the requirement of large-scale, frame-wise annotation, and 2) the complexity of two-stage approaches. To resolve these challenges, we introduce a novel semi-supervised framework by learning instance tracking networks with only a labeled image dataset and unlabeled video sequences. With an instance contrastive objective, we learn an embedding to discriminate each instance from the others. We show that even when only trained with images, the learned feature representation is robust to instance appearance variations, and is thus able to track objects steadily across frames. We further enhance the tracking capability of the embedding by learning correspondence from unlabeled videos in a self-supervised manner. In addition, we integrate this module into single-stage instance segmentation and pose estimation frameworks, which significantly reduce the computational complexity of tracking compared to two-stage networks. We conduct experiments on the YouTube-VIS and PoseTrack datasets. Without any video annotation efforts, our proposed method can achieve comparable or even better performance than most fully-supervised methods.