Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation
It addresses the problem of detecting novel objects in videos for researchers, but is incremental as it builds on existing segmentation tasks by shifting to an open-world setup.
The paper introduces UVO, a benchmark for open-world class-agnostic object segmentation in videos, which is larger and more challenging than existing datasets, with approximately 8 times more videos than DAVIS and 7 times more mask annotations per video than YouTube-VOS and YouTube-VIS.
Current state-of-the-art object detection and segmentation methods work well under the closed-world assumption. This closed-world setting assumes that the list of object categories is available during training and deployment. However, many real-world applications require detecting or segmenting novel objects, i.e., object categories never seen during training. In this paper, we present, UVO (Unidentified Video Objects), a new benchmark for open-world class-agnostic object segmentation in videos. Besides shifting the problem focus to the open-world setup, UVO is significantly larger, providing approximately 8 times more videos compared with DAVIS, and 7 times more mask (instance) annotations per video compared with YouTube-VOS and YouTube-VIS. UVO is also more challenging as it includes many videos with crowded scenes and complex background motions. We demonstrated that UVO can be used for other applications, such as object tracking and super-voxel segmentation, besides open-world object segmentation. We believe that UVo is a versatile testbed for researchers to develop novel approaches for open-world class-agnostic object segmentation, and inspires new research directions towards a more comprehensive video understanding beyond classification and detection.