GramGAN: Deep 3D Texture Synthesis From 2D Exemplars
This addresses the challenge of creating realistic 3D textures for applications in computer graphics and design, representing a novel method rather than an incremental improvement.
The paper tackles the problem of generating infinite, high-quality 3D textures from a single 2D exemplar image, achieving superior performance to previous state-of-the-art methods as confirmed by quantitative evaluations and a user study.
We present a novel texture synthesis framework, enabling the generation of infinite, high-quality 3D textures given a 2D exemplar image. Inspired by recent advances in natural texture synthesis, we train deep neural models to generate textures by non-linearly combining learned noise frequencies. To achieve a highly realistic output conditioned on an exemplar patch, we propose a novel loss function that combines ideas from both style transfer and generative adversarial networks. In particular, we train the synthesis network to match the Gram matrices of deep features from a discriminator network. In addition, we propose two architectural concepts and an extrapolation strategy that significantly improve generalization performance. In particular, we inject both model input and condition into hidden network layers by learning to scale and bias hidden activations. Quantitative and qualitative evaluations on a diverse set of exemplars motivate our design decisions and show that our system performs superior to previous state of the art. Finally, we conduct a user study that confirms the benefits of our framework.