Menghe Zhang

Menghe Zhang, Joonyeoup Kim, Yangwen Liang et al.

Accurate hand motion capture and standardized 3D representation are essential for various hand-related tasks. Collecting keypoints-only data, while efficient and cost-effective, results in low-fidelity representations and lacks surface information. Furthermore, data inconsistencies across sources challenge their integration and use. We present UniHands, a novel method for creating standardized yet personalized hand models from wild-collected keypoints from diverse sources. Unlike existing neural implicit representation methods, UniHands uses the widely-adopted parametric models MANO and NIMBLE, providing a more scalable and versatile solution. It also derives unified hand joints from the meshes, which facilitates seamless integration into various hand-related tasks. Experiments on the FreiHAND and InterHand2.6M datasets demonstrate its ability to precisely reconstruct hand mesh vertices and keypoints, effectively capturing high-degree articulation motions. Empirical studies involving nine participants show a clear preference for our unified joints over existing configurations for accuracy and naturalism (p-value 0.016).

Menghe Zhang, Jurgen Schulze, Dong Zhang

Acupuncture is a technique in which practitioners stimulate specific points on the body. Those points, called acupuncture points (or acupoints), anatomically define areas on the skin relative to specific landmarks on the body. However, mapping the acupoints to individuals could be challenging for inexperienced acupuncturists. In this project, we proposed a system to localize and visualize facial acupoints for individuals in an augmented reality (AR) context. This system combines a face alignment model and a hair segmentation model to provide dense reference points for acupoints localization in real-time (60FPS). The localization process takes the proportional bone (B-cun or skeletal) measurement method, which is commonly operated by specialists; however, in the real practice, operators sometimes find it inaccurate due to skill-related error. With this system, users, even without any skills, can locate the facial acupoints as a part of the self-training or self-treatment process.