MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints
This work addresses the challenge of producing physiologically realistic hand pose tracking for applications in computer vision and robotics, representing an incremental advancement over existing methods.
The paper tackles the problem of unnatural hand motion in existing models by integrating a musculoskeletal system with a parametric hand model, MS-MANO, and proposing a pose refinement framework, BioPR, which consistently improves baseline methods in accuracy and efficacy.
This work proposes a novel learning framework for visual hand dynamics analysis that takes into account the physiological aspects of hand motion. The existing models, which are simplified joint-actuated systems, often produce unnatural motions. To address this, we integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create a new model, MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We further propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron (MLP) network. Our evaluation of the accuracy of MS-MANO and the efficacy of the BioPR is conducted in two separate parts. The accuracy of MS-MANO is compared with MyoSuite, while the efficacy of BioPR is benchmarked against two large-scale public datasets and two recent state-of-the-art methods. The results demonstrate that our approach consistently improves the baseline methods both quantitatively and qualitatively.