Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field
This work addresses the problem of fine-grained semantic control in 3D portrait synthesis for applications like image editing, though it is incremental as it builds on existing neural radiance field methods.
The paper tackles the limitation of partial semantic editability in 3D-aware GAN methods by proposing a Compositional Neural Radiance Field (CNeRF) that divides images by semantic regions, enabling independent manipulation while maintaining high-quality 3D-consistent synthesis.
Recently 3D-aware GAN methods with neural radiance field have developed rapidly. However, current methods model the whole image as an overall neural radiance field, which limits the partial semantic editability of synthetic results. Since NeRF renders an image pixel by pixel, it is possible to split NeRF in the spatial dimension. We propose a Compositional Neural Radiance Field (CNeRF) for semantic 3D-aware portrait synthesis and manipulation. CNeRF divides the image by semantic regions and learns an independent neural radiance field for each region, and finally fuses them and renders the complete image. Thus we can manipulate the synthesized semantic regions independently, while fixing the other parts unchanged. Furthermore, CNeRF is also designed to decouple shape and texture within each semantic region. Compared to state-of-the-art 3D-aware GAN methods, our approach enables fine-grained semantic region manipulation, while maintaining high-quality 3D-consistent synthesis. The ablation studies show the effectiveness of the structure and loss function used by our method. In addition real image inversion and cartoon portrait 3D editing experiments demonstrate the application potential of our method.