Video Object Segmentation using Supervoxel-Based Gerrymandering
This work addresses the problem of segmenting objects in videos without supervision, which is crucial for applications like video editing and surveillance, and it is incremental as it builds on existing supervoxel concepts.
The paper tackles unsupervised video object segmentation by proposing a supervoxel-based gerrymandering method that uses local and non-local consensus measures to link foreground and background masks, achieving state-of-the-art performance on the DAVIS dataset, outperforming all other unsupervised methods and many supervised ones.
Pixels operate locally. Superpixels have some potential to collect information across many pixels; supervoxels have more potential by implicitly operating across time. In this paper, we explore this well established notion thoroughly analyzing how supervoxels can be used in place of and in conjunction with other means of aggregating information across space-time. Focusing on the problem of strictly unsupervised video object segmentation, we devise a method called supervoxel gerrymandering that links masks of foregroundness and backgroundness via local and non-local consensus measures. We pose and answer a series of critical questions about the ability of supervoxels to adequately sway local voting; the questions regard type and scale of supervoxels as well as local versus non-local consensus, and the questions are posed in a general way so as to impact the broader knowledge of the use of supervoxels in video understanding. We work with the DAVIS dataset and find that our analysis yields an unsupervised method that outperforms all other known unsupervised methods and even many supervised ones.