HumanGen: Generating Human Radiance Fields with Explicit Priors
This work addresses the problem of 3D human generation for applications in computer graphics and virtual reality, representing an incremental improvement by building on existing 2D and 3D methods.
The paper tackles the challenge of generating high-quality 3D human radiance fields by introducing HumanGen, which integrates explicit priors from 2D generators and 3D reconstructors to achieve detailed geometry and realistic free-view rendering, resulting in state-of-the-art performance in geometry details, texture quality, and free-view capabilities.
Recent years have witnessed the tremendous progress of 3D GANs for generating view-consistent radiance fields with photo-realism. Yet, high-quality generation of human radiance fields remains challenging, partially due to the limited human-related priors adopted in existing methods. We present HumanGen, a novel 3D human generation scheme with detailed geometry and $\text{360}^{\circ}$ realistic free-view rendering. It explicitly marries the 3D human generation with various priors from the 2D generator and 3D reconstructor of humans through the design of "anchor image". We introduce a hybrid feature representation using the anchor image to bridge the latent space of HumanGen with the existing 2D generator. We then adopt a pronged design to disentangle the generation of geometry and appearance. With the aid of the anchor image, we adapt a 3D reconstructor for fine-grained details synthesis and propose a two-stage blending scheme to boost appearance generation. Extensive experiments demonstrate our effectiveness for state-of-the-art 3D human generation regarding geometry details, texture quality, and free-view performance. Notably, HumanGen can also incorporate various off-the-shelf 2D latent editing methods, seamlessly lifting them into 3D.