MultiGO: Towards Multi-level Geometry Learning for Monocular 3D Textured Human Reconstruction
This addresses the need for more detailed 3D human reconstruction from monocular images, which is important for applications like virtual reality and animation, though it appears incremental by refining existing approaches.
The paper tackles the problem of reconstructing 3D clothed human bodies from single images, where existing methods produce inaccurate skeletons, joints, and cloth wrinkles. Their multi-level geometry learning framework achieves superior performance on out-of-distribution test sets compared to state-of-the-art methods.
This paper investigates the research task of reconstructing the 3D clothed human body from a monocular image. Due to the inherent ambiguity of single-view input, existing approaches leverage pre-trained SMPL(-X) estimation models or generative models to provide auxiliary information for human reconstruction. However, these methods capture only the general human body geometry and overlook specific geometric details, leading to inaccurate skeleton reconstruction, incorrect joint positions, and unclear cloth wrinkles. In response to these issues, we propose a multi-level geometry learning framework. Technically, we design three key components: skeleton-level enhancement, joint-level augmentation, and wrinkle-level refinement modules. Specifically, we effectively integrate the projected 3D Fourier features into a Gaussian reconstruction model, introduce perturbations to improve joint depth estimation during training, and refine the human coarse wrinkles by resembling the de-noising process of diffusion model. Extensive quantitative and qualitative experiments on two out-of-distribution test sets show the superior performance of our approach compared to state-of-the-art (SOTA) methods.