Towards an End-To-End System for Real-Time Gesture Recognition from Surface Vibrations
This work addresses the need for unobtrusive gesture recognition on everyday surfaces for smart home systems, but the results are incremental as they are demonstrated on a single surface (office desk) with a limited set of gestures.
The paper presents an end-to-end system for real-time gesture recognition from surface vibrations using piezoelectric sensors and a compact 1D-CNN, achieving high accuracy (e.g., strong user-independent performance in leave-one-subject-out cross-validation) on a dataset of 15 participants performing six gestures.
Sensing surface vibrations promise unobtrusive interaction for smart home systems by enabling gesture recognition on existing everyday surfaces without disturbing living-space design. Existing approaches typically address only parts of the processing chain, such as sensing hardware or offline gesture recognition, rather than providing an end-to-end system from surface-mounted sensors to the evaluation of the prediction model. This paper presents a custom sensor system and a configurable data-to-model pipeline for gesture recognition on a standard office desk. Our hardware enables a low-noise sensing of the vibrations using piezoelectric sensors. Building on a modular signal-processing framework, we model the full chain from continuous recordings through variable pre-processing to a model-ready dataset, and process the resulting data with compact depthwise separable 1D-CNNs. We conduct a joint search over pre-processing and model hyperparameters and identify a configuration with 8,722 parameters that uses band-pass filtering, fixed-length windows, and min-max normalization. On a self-recorded dataset with 15 participants performing six gestures this configuration achieves high accuracies across different data splitting methods, including strong user-independent performance in a leave-one-subject-out cross-validation.