Unimodal and Multimodal Sensor Fusion for Wearable Activity Recognition
It addresses the problem of accurately recognizing complex human activities for applications in wearable technology, though it appears incremental by combining existing sensing and machine learning approaches.
This Ph.D. work tackles human activity recognition by integrating inertial, pressure, and textile capacitive sensing modalities into wearable devices, achieving real-time testing with machine learning algorithms for scenarios like gesture tracking and body posture recognition.
Combining different sensing modalities with multiple positions helps form a unified perception and understanding of complex situations such as human behavior. Hence, human activity recognition (HAR) benefits from combining redundant and complementary information (Unimodal/Multimodal). Even so, it is not an easy task. It requires a multidisciplinary approach, including expertise in sensor technologies, signal processing, data fusion algorithms, and domain-specific knowledge. This Ph.D. work employs sensing modalities such as inertial, pressure (audio and atmospheric pressure), and textile capacitive sensing for HAR. The scenarios explored are gesture and hand position tracking, facial and head pattern recognition, and body posture and gesture recognition. The selected wearable devices and sensing modalities are fully integrated with machine learning-based algorithms, some of which are implemented in the embedded device, on the edge, and tested in real-time.