Towards Whole Hand and Wrist Kinematic Tracking with a Wearable A-Mode Ultrasound Probe
It addresses the need for robust, wearable, and low-power hand tracking for human-computer interaction and rehabilitation, but is incremental in applying known methods to a new sensor modality.
This work proposes a framework for robust whole-hand and wrist kinematic tracking (23 DoFs) using wearable A-mode ultrasound, achieving a 17% reduction in mean absolute error via incremental training and demonstrating on-device inference with 0.73 mJ per inference, 29.1 ms latency, and 33 mW power consumption, enabling 36 hours of continuous use.
A-mode ultrasound (US) has emerged as a promising modality for hand and wrist motion tracking. Prior works have mainly addressed static gesture classification or regression of a few degrees of freedom (DoFs), typically relying on non-wearable systems and external computing devices, and highlight the need for strategies to ensure robustness to sensor repositioning. In this work, we propose a framework for robust whole-hand and wrist kinematic tracking via wearable A-mode US using the WULPUS platform, tackling the regression of 23 DoFs directly on the probe. First, we introduce a compact (11285 parameters) multi-output convolutional neural network combined with an incremental training strategy, which improves inter-session generalization and reduces mean absolute error by more than 17% compared to a non-incremental approach. Second, we demonstrate, for the first time, the feasibility of end-to-end hand and wrist kinematic tracking entirely on-device. We deploy the model on the WULPUS nRF52832 microcontroller, achieving 0.73 mJ per inference, 29.1 ms latency, and showing the feasibility of full operation (data acquisition, online inference, and BLE streaming of results) within 33 mW, enabling up to 36 hours of continuous use and an 88% reduction in wireless bandwidth compared to raw data transmission.