VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis
This addresses the need for more flexible and controllable 3D human generation in computer vision and graphics, representing an incremental improvement over existing neural radiance field methods.
The paper tackles the problem of limited generalization and controllability in 3D-aware generative models for humans by proposing VeRi3D, a vertex-based radiance field parameterized by SMPL vertices, which enables photorealistic human image synthesis with control over camera pose, human pose, shape, and part-level editing.
Unsupervised learning of 3D-aware generative adversarial networks has lately made much progress. Some recent work demonstrates promising results of learning human generative models using neural articulated radiance fields, yet their generalization ability and controllability lag behind parametric human models, i.e., they do not perform well when generalizing to novel pose/shape and are not part controllable. To solve these problems, we propose VeRi3D, a generative human vertex-based radiance field parameterized by vertices of the parametric human template, SMPL. We map each 3D point to the local coordinate system defined on its neighboring vertices, and use the corresponding vertex feature and local coordinates for mapping it to color and density values. We demonstrate that our simple approach allows for generating photorealistic human images with free control over camera pose, human pose, shape, as well as enabling part-level editing.