MMBaT: A Multi-task Framework for mmWave-based Human Body Reconstruction and Translation Prediction
This work addresses privacy-preserving human body reconstruction for applications in adverse environments, representing an incremental improvement over existing radar-based methods.
The paper tackles human body reconstruction from noisy mmWave radar point clouds by introducing a multi-task framework that concurrently estimates body shape and predicts future translations, achieving superior performance and generalization on two public datasets compared to state-of-the-art methods.
Human body reconstruction with Millimeter Wave (mmWave) radar point clouds has gained significant interest due to its ability to work in adverse environments and its capacity to mitigate privacy concerns associated with traditional camera-based solutions. Despite pioneering efforts in this field, two challenges persist. Firstly, raw point clouds contain massive noise points, usually caused by the ambient objects and multi-path effects of Radio Frequency (RF) signals. Recent approaches typically rely on prior knowledge or elaborate preprocessing methods, limiting their applicability. Secondly, even after noise removal, the sparse and inconsistent body-related points pose an obstacle to accurate human body reconstruction. To address these challenges, we introduce mmBaT, a novel multi-task deep learning framework that concurrently estimates the human body and predicts body translations in subsequent frames to extract body-related point clouds. Our method is evaluated on two public datasets that are collected with different radar devices and noise levels. A comprehensive comparison against other state-of-the-art methods demonstrates our method has a superior reconstruction performance and generalization ability from noisy raw data, even when compared to methods provided with body-related point clouds.