Pose-variant 3D Facial Attribute Generation
This work addresses the challenge of 3D facial attribute generation for computer vision applications, representing an incremental improvement over existing 2D methods.
The paper tackles the problem of generating facial attributes from a single image in unconstrained poses by proposing a GAN-based framework that uses 3D UV texture and position maps, achieving better accuracy, photorealism, and identity preservation compared to prior methods.
We address the challenging problem of generating facial attributes using a single image in an unconstrained pose. In contrast to prior works that largely consider generation on 2D near-frontal images, we propose a GAN-based framework to generate attributes directly on a dense 3D representation given by UV texture and position maps, resulting in photorealistic, geometrically-consistent and identity-preserving outputs. Starting from a self-occluded UV texture map obtained by applying an off-the-shelf 3D reconstruction method, we propose two novel components. First, a texture completion generative adversarial network (TC-GAN) completes the partial UV texture map. Second, a 3D attribute generation GAN (3DA-GAN) synthesizes the target attribute while obtaining an appearance consistent with 3D face geometry and preserving identity. Extensive experiments on CelebA, LFW and IJB-A show that our method achieves consistently better attribute generation accuracy than prior methods, a higher degree of qualitative photorealism and preserves face identity information.