SSVOD: Semi-Supervised Video Object Detection with Sparse Annotations
This work addresses the challenge of reducing annotation costs for video object detection, which is important for applications in video analysis, but it is incremental as it builds on existing semi-supervised techniques by adapting them to videos.
The paper tackles the problem of expensive and redundant frame annotations in video object detection by proposing SSVOD, a semi-supervised framework that exploits motion dynamics and uses sparse annotations, achieving significant performance improvements on datasets like ImageNet-VID, Epic-KITCHENS, and YouTube-VIS.
Despite significant progress in semi-supervised learning for image object detection, several key issues are yet to be addressed for video object detection: (1) Achieving good performance for supervised video object detection greatly depends on the availability of annotated frames. (2) Despite having large inter-frame correlations in a video, collecting annotations for a large number of frames per video is expensive, time-consuming, and often redundant. (3) Existing semi-supervised techniques on static images can hardly exploit the temporal motion dynamics inherently present in videos. In this paper, we introduce SSVOD, an end-to-end semi-supervised video object detection framework that exploits motion dynamics of videos to utilize large-scale unlabeled frames with sparse annotations. To selectively assemble robust pseudo-labels across groups of frames, we introduce \textit{flow-warped predictions} from nearby frames for temporal-consistency estimation. In particular, we introduce cross-IoU and cross-divergence based selection methods over a set of estimated predictions to include robust pseudo-labels for bounding boxes and class labels, respectively. To strike a balance between confirmation bias and uncertainty noise in pseudo-labels, we propose confidence threshold based combination of hard and soft pseudo-labels. Our method achieves significant performance improvements over existing methods on ImageNet-VID, Epic-KITCHENS, and YouTube-VIS datasets. Code and pre-trained models will be released.