MFT: Long-Term Tracking of Every Pixel
This addresses the need for efficient and robust dense tracking in computer vision, though it appears incremental as it builds on existing optical flow and CNN techniques.
The paper tackles the problem of dense, long-term pixel-level tracking by proposing MFT, a method that uses multi-flow optical flows and CNN-based reliability selection, achieving competitive performance on the TAP-Vid benchmark with significant speed improvements over state-of-the-art methods.
We propose MFT -- Multi-Flow dense Tracker -- a novel method for dense, pixel-level, long-term tracking. The approach exploits optical flows estimated not only between consecutive frames, but also for pairs of frames at logarithmically spaced intervals. It selects the most reliable sequence of flows on the basis of estimates of its geometric accuracy and the probability of occlusion, both provided by a pre-trained CNN. We show that MFT achieves competitive performance on the TAP-Vid benchmark, outperforming baselines by a significant margin, and tracking densely orders of magnitude faster than the state-of-the-art point-tracking methods. The method is insensitive to medium-length occlusions and it is robustified by estimating flow with respect to the reference frame, which reduces drift.