PaintHuman: Towards High-fidelity Text-to-3D Human Texturing via Denoised Score Distillation
This addresses the challenge of detailed text-to-3D human generation for applications in graphics and AI, representing an incremental improvement over prior diffusion-based techniques.
The paper tackles the problem of generating high-fidelity 3D human textures from text, where existing methods like Score Distillation Sampling (SDS) produce over-smoothed and inconsistent results. It proposes PaintHuman, which introduces Denoised Score Distillation (DSD) and depth guidance to achieve improved texture quality, validated through experiments against state-of-the-art methods.
Recent advances in zero-shot text-to-3D human generation, which employ the human model prior (eg, SMPL) or Score Distillation Sampling (SDS) with pre-trained text-to-image diffusion models, have been groundbreaking. However, SDS may provide inaccurate gradient directions under the weak diffusion guidance, as it tends to produce over-smoothed results and generate body textures that are inconsistent with the detailed mesh geometry. Therefore, directly leverage existing strategies for high-fidelity text-to-3D human texturing is challenging. In this work, we propose a model called PaintHuman to addresses the challenges from two aspects. We first propose a novel score function, Denoised Score Distillation (DSD), which directly modifies the SDS by introducing negative gradient components to iteratively correct the gradient direction and generate high-quality textures. In addition, we use the depth map as a geometric guidance to ensure the texture is semantically aligned to human mesh surfaces. To guarantee the quality of rendered results, we employ geometry-aware networks to predict surface materials and render realistic human textures. Extensive experiments, benchmarked against state-of-the-art methods, validate the efficacy of our approach.