DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos
This work addresses the lack of annotated real-world video data for training keypoint trackers, which is a bottleneck for applications in autonomous driving and similar domains, representing an incremental advancement in dataset creation.
The paper tackles the problem of long-range keypoint tracking in real-world videos by introducing DriveTrack, a benchmark and data generation framework that automatically annotates point tracks on autonomous driving datasets, resulting in a dataset with 1 billion point tracks and improving tracker accuracy by up to 7%.
This paper presents DriveTrack, a new benchmark and data generation framework for long-range keypoint tracking in real-world videos. DriveTrack is motivated by the observation that the accuracy of state-of-the-art trackers depends strongly on visual attributes around the selected keypoints, such as texture and lighting. The problem is that these artifacts are especially pronounced in real-world videos, but these trackers are unable to train on such scenes due to a dearth of annotations. DriveTrack bridges this gap by building a framework to automatically annotate point tracks on autonomous driving datasets. We release a dataset consisting of 1 billion point tracks across 24 hours of video, which is seven orders of magnitude greater than prior real-world benchmarks and on par with the scale of synthetic benchmarks. DriveTrack unlocks new use cases for point tracking in real-world videos. First, we show that fine-tuning keypoint trackers on DriveTrack improves accuracy on real-world scenes by up to 7%. Second, we analyze the sensitivity of trackers to visual artifacts in real scenes and motivate the idea of running assistive keypoint selectors alongside trackers.