3D-GANTex: 3D Face Reconstruction with StyleGAN3-based Multi-View Images and 3DDFA based Mesh Generation
This work addresses a domain-specific problem in 3D face reconstruction for computer vision applications, but it appears incremental as it builds on existing techniques like StyleGAN and 3DDFA.
The paper tackled the ill-posed problem of geometry and texture estimation from a single face image, especially under different angles, by introducing a method that uses StyleGAN and 3D Morphable Models to generate multi-view faces and estimate a 3D mesh with high-resolution texture, resulting in high-quality meshes with near-accurate texture representation.
Geometry and texture estimation from a single face image is an ill-posed problem since there is very little information to work with. The problem further escalates when the face is rotated at a different angle. This paper tries to tackle this problem by introducing a novel method for texture estimation from a single image by first using StyleGAN and 3D Morphable Models. The method begins by generating multi-view faces using the latent space of GAN. Then 3DDFA trained on 3DMM estimates a 3D face mesh as well as a high-resolution texture map that is consistent with the estimated face shape. The result shows that the generated mesh is of high quality with near to accurate texture representation.