Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagation
This addresses the annotation scarcity issue in 3D hand pose estimation for real-world applications, but it is incremental as it builds on semi-supervised methods.
The paper tackles the problem of limited 3D annotations for hand shape and pose estimation by proposing a Pose Alignment network to propagate labels from labeled to unlabeled frames in videos, improving pose estimation accuracy without fine-tuning on unseen data.
To obtain 3D annotations, we are restricted to controlled environments or synthetic datasets, leading us to 3D datasets with less generalizability to real-world scenarios. To tackle this issue in the context of semi-supervised 3D hand shape and pose estimation, we propose the Pose Alignment network to propagate 3D annotations from labelled frames to nearby unlabelled frames in sparsely annotated videos. We show that incorporating the alignment supervision on pairs of labelled-unlabelled frames allows us to improve the pose estimation accuracy. Besides, we show that the proposed Pose Alignment network can effectively propagate annotations on unseen sparsely labelled videos without fine-tuning.