PoseSync: Robust pose based video synchronization
This work addresses video synchronization for domains such as sports and choreography, but it appears incremental as it builds on existing pose detection and DTW methods.
The authors tackled the problem of synchronizing videos based on human pose for applications like performance evaluation, proposing an end-to-end pipeline that uses pose detection and Dynamic Time Warping to achieve scale and shift invariant matching.
Pose based video sychronization can have applications in multiple domains such as gameplay performance evaluation, choreography or guiding athletes. The subject's actions could be compared and evaluated against those performed by professionals side by side. In this paper, we propose an end to end pipeline for synchronizing videos based on pose. The first step crops the region where the person present in the image followed by pose detection on the cropped image. This is followed by application of Dynamic Time Warping(DTW) on angle/ distance measures between the pose keypoints leading to a scale and shift invariant pose matching pipeline.