Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance
This work addresses the challenge of 3D hand reconstruction for computer vision applications by reducing reliance on costly 3D data, though it is incremental as it builds on existing weakly-supervised approaches.
The paper tackles the problem of monocular 3D hand reconstruction without expensive 3D supervision by introducing a weakly-supervised method that leverages hand knowledge from biomechanics, anatomy, and physics, and incorporates uncertainty modeling, resulting in a 21% performance improvement on the FreiHAND dataset.
Fully-supervised monocular 3D hand reconstruction is often difficult because capturing the requisite 3D data entails deploying specialized equipment in a controlled environment. We introduce a weakly-supervised method that avoids such requirements by leveraging fundamental principles well-established in the understanding of the human hand's unique structure and functionality. Specifically, we systematically study hand knowledge from different sources, including biomechanics, functional anatomy, and physics. We effectively incorporate these valuable foundational insights into 3D hand reconstruction models through an appropriate set of differentiable training losses. This enables training solely with readily-obtainable 2D hand landmark annotations and eliminates the need for expensive 3D supervision. Moreover, we explicitly model the uncertainty that is inherent in image observations. We enhance the training process by exploiting a simple yet effective Negative Log Likelihood (NLL) loss that incorporates uncertainty into the loss function. Through extensive experiments, we demonstrate that our method significantly outperforms state-of-the-art weakly-supervised methods. For example, our method achieves nearly a 21\% performance improvement on the widely adopted FreiHAND dataset.