SMPLitex: A Generative Model and Dataset for 3D Human Texture Estimation from Single Image
This addresses the challenge of 3D human texture estimation for computer vision and graphics applications, representing a novel method for a known bottleneck.
The authors tackled the problem of estimating and manipulating complete 3D human appearance from a single image, resulting in SMPLitex, which significantly outperforms existing methods for human texture estimation and enables tasks like editing and synthesis.
We propose SMPLitex, a method for estimating and manipulating the complete 3D appearance of humans captured from a single image. SMPLitex builds upon the recently proposed generative models for 2D images, and extends their use to the 3D domain through pixel-to-surface correspondences computed on the input image. To this end, we first train a generative model for complete 3D human appearance, and then fit it into the input image by conditioning the generative model to the visible parts of the subject. Furthermore, we propose a new dataset of high-quality human textures built by sampling SMPLitex conditioned on subject descriptions and images. We quantitatively and qualitatively evaluate our method in 3 publicly available datasets, demonstrating that SMPLitex significantly outperforms existing methods for human texture estimation while allowing for a wider variety of tasks such as editing, synthesis, and manipulation