Disentangling 3D Attributes from a Single 2D Image: Human Pose, Shape and Garment
This addresses the challenge of interpretable 3D human reconstruction from 2D images for applications in visual manipulation, though it appears incremental as it builds on existing encoder-decoder and disentanglement methods.
The paper tackles the problem of extracting disentangled 3D attributes (human pose, shape, and garment) from a single 2D image, achieving cross-domain disentanglement for this under-constrained task and demonstrating the ability to transfer these attributes in 3D reconstruction on virtual data.
For visual manipulation tasks, we aim to represent image content with semantically meaningful features. However, learning implicit representations from images often lacks interpretability, especially when attributes are intertwined. We focus on the challenging task of extracting disentangled 3D attributes only from 2D image data. Specifically, we focus on human appearance and learn implicit pose, shape and garment representations of dressed humans from RGB images. Our method learns an embedding with disentangled latent representations of these three image properties and enables meaningful re-assembling of features and property control through a 2D-to-3D encoder-decoder structure. The 3D model is inferred solely from the feature map in the learned embedding space. To the best of our knowledge, our method is the first to achieve cross-domain disentanglement for this highly under-constrained problem. We qualitatively and quantitatively demonstrate our framework's ability to transfer pose, shape, and garments in 3D reconstruction on virtual data and show how an implicit shape loss can benefit the model's ability to recover fine-grained reconstruction details.