Texture Generation Using A Graph Generative Adversarial Network And Differentiable Rendering
This work addresses the challenge of efficient 3D texture generation for applications like 3D simulations, offering a novel unsupervised method that avoids costly data and computational requirements.
The paper tackles the problem of generating novel photo-realistic textures for 3D objects without requiring expensive 3D part segmentation or forward passes for viewpoint changes, and demonstrates that their graph generative adversarial network (GGAN) achieves significantly better texture generation quality compared to state-of-the-art methods, as measured by Frechet inception distance.
Novel photo-realistic texture synthesis is an important task for generating novel scenes, including asset generation for 3D simulations. However, to date, these methods predominantly generate textured objects in 2D space. If we rely on 2D object generation, then we need to make a computationally expensive forward pass each time we change the camera viewpoint or lighting. Recent work that can generate textures in 3D requires 3D component segmentation that is expensive to acquire. In this work, we present a novel conditional generative architecture that we call a graph generative adversarial network (GGAN) that can generate textures in 3D by learning object component information in an unsupervised way. In this framework, we do not need an expensive forward pass whenever the camera viewpoint or lighting changes, and we do not need expensive 3D part information for training, yet the model can generalize to unseen 3D meshes and generate appropriate novel 3D textures. We compare this approach against state-of-the-art texture generation methods and demonstrate that the GGAN obtains significantly better texture generation quality (according to Frechet inception distance). We release our model source code as open source.