Detect or Track: Towards Cost-Effective Video Object Detection/Tracking
This work addresses the efficiency-accuracy trade-off in video object detection/tracking, offering a cost-effective solution for real-time applications.
The paper tackles the problem of balancing detection and tracking for video object analysis under a time budget, proposing a scheduler network that outperforms frame skipping and flow-based baselines on the ImageNet VID dataset.
State-of-the-art object detectors and trackers are developing fast. Trackers are in general more efficient than detectors but bear the risk of drifting. A question is hence raised -- how to improve the accuracy of video object detection/tracking by utilizing the existing detectors and trackers within a given time budget? A baseline is frame skipping -- detecting every N-th frames and tracking for the frames in between. This baseline, however, is suboptimal since the detection frequency should depend on the tracking quality. To this end, we propose a scheduler network, which determines to detect or track at a certain frame, as a generalization of Siamese trackers. Although being light-weight and simple in structure, the scheduler network is more effective than the frame skipping baselines and flow-based approaches, as validated on ImageNet VID dataset in video object detection/tracking.