Overcoming the Trade-off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction
This work addresses the problem of generating accurate and plausible 3D hand shapes for applications like two-hand and hand-object interaction scenarios, representing an incremental improvement by combining existing methods.
The paper tackled the trade-off between accuracy and plausibility in 3D hand shape reconstruction by introducing a weakly-supervised framework that integrates non-parametric mesh fitting with the MANO model, resulting in well-aligned and high-quality 3D meshes.
Direct mesh fitting for 3D hand shape reconstruction is highly accurate. However, the reconstructed meshes are prone to artifacts and do not appear as plausible hand shapes. Conversely, parametric models like MANO ensure plausible hand shapes but are not as accurate as the non-parametric methods. In this work, we introduce a novel weakly-supervised hand shape estimation framework that integrates non-parametric mesh fitting with MANO model in an end-to-end fashion. Our joint model overcomes the tradeoff in accuracy and plausibility to yield well-aligned and high-quality 3D meshes, especially in challenging two-hand and hand-object interaction scenarios.