Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimation
This work provides an incremental improvement in 3D hand pose estimation accuracy for researchers and applications in whole-body 3D human mesh estimation.
This paper addresses the challenge of accurate 3D hand pose estimation within whole-body 3D human mesh reconstruction. The authors propose Hand4Whole, which improves 3D wrist prediction by considering the human kinematic chain and uses specific hand MCP joint features. It also discards body features for 3D finger rotation prediction, leading to significantly better 3D hand results compared to prior methods.
Whole-body 3D human mesh estimation aims to reconstruct the 3D human body, hands, and face simultaneously. Although several methods have been proposed, accurate prediction of 3D hands, which consist of 3D wrist and fingers, still remains challenging due to two reasons. First, the human kinematic chain has not been carefully considered when predicting the 3D wrists. Second, previous works utilize body features for the 3D fingers, where the body feature barely contains finger information. To resolve the limitations, we present Hand4Whole, which has two strong points over previous works. First, we design Pose2Pose, a module that utilizes joint features for 3D joint rotations. Using Pose2Pose, Hand4Whole utilizes hand MCP joint features to predict 3D wrists as MCP joints largely contribute to 3D wrist rotations in the human kinematic chain. Second, Hand4Whole discards the body feature when predicting 3D finger rotations. Our Hand4Whole is trained in an end-to-end manner and produces much better 3D hand results than previous whole-body 3D human mesh estimation methods. The codes are available here at https://github.com/mks0601/Hand4Whole_RELEASE.