Video Region Annotation with Sparse Bounding Boxes
This addresses the labor-intensive need for dense video annotations in tasks like segmentation, offering a more efficient solution for video analysis applications.
The paper tackled the problem of generating detailed region boundaries for all video frames from sparse bounding box annotations, achieving effective results with a Volumetric Graph Convolutional Network (VGCN) that outperformed existing solutions in experiments on real and synthetic datasets.
Video analysis has been moving towards more detailed interpretation (e.g. segmentation) with encouraging progresses. These tasks, however, increasingly rely on densely annotated training data both in space and time. Since such annotation is labour-intensive, few densely annotated video data with detailed region boundaries exist. This work aims to resolve this dilemma by learning to automatically generate region boundaries for all frames of a video from sparsely annotated bounding boxes of target regions. We achieve this with a Volumetric Graph Convolutional Network (VGCN), which learns to iteratively find keypoints on the region boundaries using the spatio-temporal volume of surrounding appearance and motion. The global optimization of VGCN makes it significantly stronger and generalize better than existing solutions. Experimental results using two latest datasets (one real and one synthetic), including ablation studies, demonstrate the effectiveness and superiority of our method.