Unsupervised Video Object Segmentation with Distractor-Aware Online Adaptation
This addresses the problem of segmenting objects in videos without prior information, particularly when multiple objects interact, for applications in video analysis, but it is incremental as it builds on existing instance segmentation and motion tracking techniques.
The paper tackles unsupervised video object segmentation by proposing a distractor-aware online adaptation (DOA) method that models spatial-temporal consistency and uses positive, negative, and hard negative masks to update segmentation, achieving state-of-the-art results on DAVIS 2016 and FBMS-59 datasets.
Unsupervised video object segmentation is a crucial application in video analysis without knowing any prior information about the objects. It becomes tremendously challenging when multiple objects occur and interact in a given video clip. In this paper, a novel unsupervised video object segmentation approach via distractor-aware online adaptation (DOA) is proposed. DOA models spatial-temporal consistency in video sequences by capturing background dependencies from adjacent frames. Instance proposals are generated by the instance segmentation network for each frame and then selected by motion information as hard negatives if they exist and positives. To adopt high-quality hard negatives, the block matching algorithm is then applied to preceding frames to track the associated hard negatives. General negatives are also introduced in case that there are no hard negatives in the sequence and experiments demonstrate both kinds of negatives (distractors) are complementary. Finally, we conduct DOA using the positive, negative, and hard negative masks to update the foreground/background segmentation. The proposed approach achieves state-of-the-art results on two benchmark datasets, DAVIS 2016 and FBMS-59 datasets.