Towards Robust RGB-D Human Mesh Recovery
This work addresses the under-constrained nature of RGB-based human mesh recovery for applications in computer vision, though it appears incremental by building on existing parametric models like SMPL.
The paper tackles the problem of human pose estimation by proposing a method that uses RGB-D data to estimate a parametric human mesh model, achieving improved robustness over RGB-only techniques through innovations in data fusion, constraint generation, and depth ranking.
We consider the problem of human pose estimation. While much recent work has focused on the RGB domain, these techniques are inherently under-constrained since there can be many 3D configurations that explain the same 2D projection. To this end, we propose a new method that uses RGB-D data to estimate a parametric human mesh model. Our key innovations include (a) the design of a new dynamic data fusion module that facilitates learning with a combination of RGB-only and RGB-D datasets, (b) a new constraint generator module that provides SMPL supervisory signals when explicit SMPL annotations are not available, and (c) the design of a new depth ranking learning objective, all of which enable principled model training with RGB-D data. We conduct extensive experiments on a variety of RGB-D datasets to demonstrate efficacy.