WatchHAR: Real-time On-device Human Activity Recognition System for Smartwatches
This advances on-device activity recognition for smartwatch users, addressing privacy and latency issues, though it is incremental in optimizing existing components.
The paper tackles the problem of real-time human activity recognition on smartwatches by presenting WatchHAR, an on-device system that uses audio and inertial sensors, achieving over 90% accuracy across 25+ activity classes with processing times as low as 9.3 ms.
Despite advances in practical and multimodal fine-grained Human Activity Recognition (HAR), a system that runs entirely on smartwatches in unconstrained environments remains elusive. We present WatchHAR, an audio and inertial-based HAR system that operates fully on smartwatches, addressing privacy and latency issues associated with external data processing. By optimizing each component of the pipeline, WatchHAR achieves compounding performance gains. We introduce a novel architecture that unifies sensor data preprocessing and inference into an end-to-end trainable module, achieving 5x faster processing while maintaining over 90% accuracy across more than 25 activity classes. WatchHAR outperforms state-of-the-art models for event detection and activity classification while running directly on the smartwatch, achieving 9.3 ms processing time for activity event detection and 11.8 ms for multimodal activity classification. This research advances on-device activity recognition, realizing smartwatches' potential as standalone, privacy-aware, and minimally-invasive continuous activity tracking devices.