Acoustic Sensing-based Hand Gesture Detection for Wearable Device Interaction
This enables low-cost, intuitive interaction for smartwatch and AR/VR users, though it is incremental as it builds on existing acoustic sensing methods.
The paper tackles hand gesture recognition for wearable devices by using bone-conducted sound from finger movements, achieving accuracies of 90.13% in quiet and 85.79% in noisy environments.
Hand gesture recognition attracts great attention for interaction since it is intuitive and natural to perform. In this paper, we explore a novel method for interaction by using bone-conducted sound generated by finger movements while performing gestures. We design a set of gestures that generate unique sound features, and capture the resulting sound from the wrist using a commodity microphone. Next, we design a sound event detector and a recognition model to classify the gestures. Our system achieves an overall accuracy of 90.13% in quiet environments and 85.79% under noisy conditions. This promising technology can be deployed on existing smartwatches as a low power service at no additional cost, and can be used for interaction in augmented and virtual reality applications.