Structure-Aware 3D Hourglass Network for Hand Pose Estimation from Single Depth Image
This work addresses hand pose estimation for applications like human-computer interaction, but it is incremental as it builds on existing hourglass networks with 3D adaptations and skeleton constraints.
The paper tackles hand pose estimation from a single depth image by proposing a structure-aware 3D hourglass network that directly regresses 3D heatmaps for joints, achieving state-of-the-art results with mean joint errors of 7.4 mm on MSRA and 8.9 mm on NYU datasets.
In this paper, we propose a novel structure-aware 3D hourglass network for hand pose estimation from a single depth image, which achieves state-of-the-art results on MSRA and NYU datasets. Compared to existing works that perform image-to-coordination regression, our network takes 3D voxel as input and directly regresses 3D heatmap for each joint. To be specific, we use hourglass network as our backbone network and modify it into 3D form. We explicitly model tree-like finger bone into the network as well as in the loss function in an end-to-end manner, in order to take the skeleton constraints into consideration. Final estimation can then be easily obtained from voxel density map with simple post-processing. Experimental results show that the proposed structure-aware 3D hourglass network is able to achieve a mean joint error of 7.4 mm in MSRA and 8.9 mm in NYU datasets, respectively.