Recovering Spatiotemporal Correspondence between Deformable Objects by Exploiting Consistent Foreground Motion in Video
This addresses the challenge of mapping deformable objects in videos for applications like wildlife monitoring, though it appears incremental as it builds on motion-based methods rather than introducing a new paradigm.
The paper tackles the problem of automatically recovering spatiotemporal correspondences between deformable objects in unstructured videos, such as animals in the wild, by exploiting consistency in object motion. It achieves high accuracy in aligning thousands of frame pairs on a dataset of tiger and horse videos, outperforming the SIFT Flow algorithm.
Given unstructured videos of deformable objects, we automatically recover spatiotemporal correspondences to map one object to another (such as animals in the wild). While traditional methods based on appearance fail in such challenging conditions, we exploit consistency in object motion between instances. Our approach discovers pairs of short video intervals where the object moves in a consistent manner and uses these candidates as seeds for spatial alignment. We model the spatial correspondence between the point trajectories on the object in one interval to those in the other using a time-varying Thin Plate Spline deformation model. On a large dataset of tiger and horse videos, our method automatically aligns thousands of pairs of frames to a high accuracy, and outperforms the popular SIFT Flow algorithm.