PresSim: An End-to-end Framework for Dynamic Ground Pressure Profile Generation from Monocular Videos Using Physics-based 3D Simulation
This work addresses the need for unobtrusive pervasive sensing in human activity recognition by reducing data collection effort, though it is incremental as it builds on existing computer vision and simulation methods.
The paper tackles the problem of resource-intensive data collection for human activity recognition by introducing PresSim, an end-to-end framework that synthesizes ground pressure sensor data from monocular videos using physics-based 3D simulation, validated with a pressure-sensing mat (80x28 resolution) on nine participants performing yoga sequences.
Ground pressure exerted by the human body is a valuable source of information for human activity recognition (HAR) in unobtrusive pervasive sensing. While data collection from pressure sensors to develop HAR solutions requires significant resources and effort, we present a novel end-to-end framework, PresSim, to synthesize sensor data from videos of human activities to reduce such effort significantly. PresSim adopts a 3-stage process: first, extract the 3D activity information from videos with computer vision architectures; then simulate the floor mesh deformation profiles based on the 3D activity information and gravity-included physics simulation; lastly, generate the simulated pressure sensor data with deep learning models. We explored two approaches for the 3D activity information: inverse kinematics with mesh re-targeting, and volumetric pose and shape estimation. We validated PresSim with an experimental setup with a monocular camera to provide input and a pressure-sensing fitness mat (80x28 spatial resolution) to provide the sensor ground truth, where nine participants performed a set of predefined yoga sequences.