Detailed Avatar Recovery from Single Image
This addresses the need for high-fidelity avatar creation in applications like virtual reality or gaming, though it is incremental as it builds on existing parametric models.
The paper tackles the problem of recovering detailed 3D avatars from single images, which is challenging due to variations in shape, pose, and texture, and it achieves better accuracy than prior methods in terms of 2D IoU and 3D metric distance.
This paper presents a novel framework to recover \emph{detailed} avatar from a single image. It is a challenging task due to factors such as variations in human shapes, body poses, texture, and viewpoints. Prior methods typically attempt to recover the human body shape using a parametric-based template that lacks the surface details. As such resulting body shape appears to be without clothing. In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and per-pixel shading information. Our method can restore detailed human body shapes with complete textures beyond skinned models. Experiments demonstrate that our method has outperformed previous state-of-the-art approaches, achieving better accuracy in terms of both 2D IoU number and 3D metric distance.