mmDEAR: mmWave Point Cloud Density Enhancement for Accurate Human Body Reconstruction
This work addresses privacy-friendly human body reconstruction for applications like surveillance or healthcare, but it is incremental as it builds on existing models with enhancements.
The paper tackles the problem of sparse mmWave radar point clouds limiting human body reconstruction accuracy by proposing a two-stage deep learning framework that densifies the data and refines SMPL parameters, resulting in outperforming state-of-the-art methods on multiple datasets.
Millimeter-wave (mmWave) radar offers robust sensing capabilities in diverse environments, making it a highly promising solution for human body reconstruction due to its privacy-friendly and non-intrusive nature. However, the significant sparsity of mmWave point clouds limits the estimation accuracy. To overcome this challenge, we propose a two-stage deep learning framework that enhances mmWave point clouds and improves human body reconstruction accuracy. Our method includes a mmWave point cloud enhancement module that densifies the raw data by leveraging temporal features and a multi-stage completion network, followed by a 2D-3D fusion module that extracts both 2D and 3D motion features to refine SMPL parameters. The mmWave point cloud enhancement module learns the detailed shape and posture information from 2D human masks in single-view images. However, image-based supervision is involved only during the training phase, and the inference relies solely on sparse point clouds to maintain privacy. Experiments on multiple datasets demonstrate that our approach outperforms state-of-the-art methods, with the enhanced point clouds further improving performance when integrated into existing models.