Weakly-Supervised Photo-realistic Texture Generation for 3D Face Reconstruction
This addresses the lack of focus on texture generation in 3D face reconstruction, offering a novel approach for applications in graphics and VR.
The paper tackles the problem of generating high-fidelity 3D face textures from a single image, proposing a weakly-supervised model that achieves photo-realistic results by combining a UV sampler and generator with partial discriminators.
Although much progress has been made recently in 3D face reconstruction, most previous work has been devoted to predicting accurate and fine-grained 3D shapes. In contrast, relatively little work has focused on generating high-fidelity face textures. Compared with the prosperity of photo-realistic 2D face image generation, high-fidelity 3D face texture generation has yet to be studied. In this paper, we proposed a novel UV map generation model that predicts the UV map from a single face image. The model consists of a UV sampler and a UV generator. By selectively sampling the input face image's pixels and adjusting their relative locations, the UV sampler generates an incomplete UV map that could faithfully reconstruct the original face. Missing textures in the incomplete UV map are further full-filled by the UV generator. The training is based on pseudo ground truth blended by the 3DMM texture and the input face texture, thus weakly supervised. To deal with the artifacts in the imperfect pseudo UV map, multiple partial UV map discriminators are leveraged.