Monocular 3D Hand Mesh Recovery via Dual Noise Estimation
This addresses the challenge of accurate 3D hand mesh recovery for applications like VR/AR, though it is incremental as it builds on existing parametric models.
The paper tackles the problem of generating well-aligned 3D hand meshes from monocular images by introducing a dual noise estimation method, which improves baseline performance by over 10% and achieves state-of-the-art results on the Interhand2.6M dataset.
Current parametric models have made notable progress in 3D hand pose and shape estimation. However, due to the fixed hand topology and complex hand poses, current models are hard to generate meshes that are aligned with the image well. To tackle this issue, we introduce a dual noise estimation method in this paper. Given a single-view image as input, we first adopt a baseline parametric regressor to obtain the coarse hand meshes. We assume the mesh vertices and their image-plane projections are noisy, and can be associated in a unified probabilistic model. We then learn the distributions of noise to refine mesh vertices and their projections. The refined vertices are further utilized to refine camera parameters in a closed-form manner. Consequently, our method obtains well-aligned and high-quality 3D hand meshes. Extensive experiments on the large-scale Interhand2.6M dataset demonstrate that the proposed method not only improves the performance of its baseline by more than 10$\%$ but also achieves state-of-the-art performance. Project page: \url{https://github.com/hanhuili/DNE4Hand}.