HumanMeshNet: Polygonal Mesh Recovery of Humans
This addresses the problem of efficient and accurate 3D human modeling for applications like virtual reality and animation, though it is incremental as it builds on existing mesh-based approaches.
The paper tackles 3D human body reconstruction from monocular images by proposing HumanMeshNet, which regresses mesh vertices with regularization from skeletal locations and mesh topology, achieving comparable performance to state-of-the-art methods with lower computational cost and enabling real-time reconstructions on three datasets.
3D Human Body Reconstruction from a monocular image is an important problem in computer vision with applications in virtual and augmented reality platforms, animation industry, en-commerce domain, etc. While several of the existing works formulate it as a volumetric or parametric learning with complex and indirect reliance on re-projections of the mesh, we would like to focus on implicitly learning the mesh representation. To that end, we propose a novel model, HumanMeshNet, that regresses a template mesh's vertices, as well as receives a regularization by the 3D skeletal locations in a multi-branch, multi-task setup. The image to mesh vertex regression is further regularized by the neighborhood constraint imposed by mesh topology ensuring smooth surface reconstruction. The proposed paradigm can theoretically learn local surface deformations induced by body shape variations and can therefore learn high-resolution meshes going ahead. We show comparable performance with SoA (in terms of surface and joint error) with far lesser computational complexity, modeling cost and therefore real-time reconstructions on three publicly available datasets. We also show the generalizability of the proposed paradigm for a similar task of predicting hand mesh models. Given these initial results, we would like to exploit the mesh topology in an explicit manner going ahead.