BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networks
This work addresses a specific problem in computer vision for applications requiring detailed 3D hand modeling, but it is incremental as it builds on existing trends in hand estimation with a novel network design.
The paper tackles the problem of recovering a full 3D hand mesh from a single RGB image, which is challenging due to the need to estimate joint rotations and shape parameters. The result is that their model, BiHand, achieves superior accuracy on public benchmarks like RHD and STB, producing appealing meshes even in severe conditions.
3D hand estimation has been a long-standing research topic in computer vision. A recent trend aims not only to estimate the 3D hand joint locations but also to recover the mesh model. However, achieving those goals from a single RGB image remains challenging. In this paper, we introduce an end-to-end learnable model, BiHand, which consists of three cascaded stages, namely 2D seeding stage, 3D lifting stage, and mesh generation stage. At the output of BiHand, the full hand mesh will be recovered using the joint rotations and shape parameters predicted from the network. Inside each stage, BiHand adopts a novel bisecting design which allows the networks to encapsulate two closely related information (e.g. 2D keypoints and silhouette in 2D seeding stage, 3D joints, and depth map in 3D lifting stage, joint rotations and shape parameters in the mesh generation stage) in a single forward pass. As the information represents different geometry or structure details, bisecting the data flow can facilitate optimization and increase robustness. For quantitative evaluation, we conduct experiments on two public benchmarks, namely the Rendered Hand Dataset (RHD) and the Stereo Hand Pose Tracking Benchmark (STB). Extensive experiments show that our model can achieve superior accuracy in comparison with state-of-the-art methods, and can produce appealing 3D hand meshes in several severe conditions.