TetraTSDF: 3D human reconstruction from a single image with a tetrahedral outer shell
This addresses the challenge of detailed 3D human reconstruction from single images for applications in computer vision and graphics, though it appears incremental as it builds on prior CNN-based methods with a new representation.
The paper tackles the problem of reconstructing detailed 3D human shapes from single images, particularly for people wearing loose clothes, by proposing a compact and dense tetrahedral TSDF model and a part connection network, achieving accurate reconstructions as demonstrated in results.
Recovering the 3D shape of a person from its 2D appearance is ill-posed due to ambiguities. Nevertheless, with the help of convolutional neural networks (CNN) and prior knowledge on the 3D human body, it is possible to overcome such ambiguities to recover detailed 3D shapes of human bodies from single images. Current solutions, however, fail to reconstruct all the details of a person wearing loose clothes. This is because of either (a) huge memory requirement that cannot be maintained even on modern GPUs or (b) the compact 3D representation that cannot encode all the details. In this paper, we propose the tetrahedral outer shell volumetric truncated signed distance function (TetraTSDF) model for the human body, and its corresponding part connection network (PCN) for 3D human body shape regression. Our proposed model is compact, dense, accurate, and yet well suited for CNN-based regression task. Our proposed PCN allows us to learn the distribution of the TSDF in the tetrahedral volume from a single image in an end-to-end manner. Results show that our proposed method allows to reconstruct detailed shapes of humans wearing loose clothes from single RGB images.