Multi-view Inversion for 3D-aware Generative Adversarial Networks
This work addresses the limitation of single-view 3D GAN inversion for human heads, offering improved reconstruction for applications like animation or virtual reality, though it is incremental as it builds on existing state-of-the-art techniques.
The paper tackles the problem of reconstructing 3D head models from single frontal images by proposing a method that uses multiple views or dynamic videos for consistent inversion, resulting in significant enhancements in geometric accuracy and image quality, especially for wide viewing angles.
Current 3D GAN inversion methods for human heads typically use only one single frontal image to reconstruct the whole 3D head model. This leaves out meaningful information when multi-view data or dynamic videos are available. Our method builds on existing state-of-the-art 3D GAN inversion techniques to allow for consistent and simultaneous inversion of multiple views of the same subject. We employ a multi-latent extension to handle inconsistencies present in dynamic face videos to re-synthesize consistent 3D representations from the sequence. As our method uses additional information about the target subject, we observe significant enhancements in both geometric accuracy and image quality, particularly when rendering from wide viewing angles. Moreover, we demonstrate the editability of our inverted 3D renderings, which distinguishes them from NeRF-based scene reconstructions.