Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis
This method enables efficient, text-guided texture synthesis for applications like 3D rendering and texture transfer, but it is incremental as it builds on existing diffusion models.
The authors tackled the problem of generating arbitrarily large texture images from text prompts by fine-tuning a diffusion model on a single texture and using a score aggregation strategy, achieving high-resolution synthesis on a single GPU.
We present Infinite Texture, a method for generating arbitrarily large texture images from a text prompt. Our approach fine-tunes a diffusion model on a single texture, and learns to embed that statistical distribution in the output domain of the model. We seed this fine-tuning process with a sample texture patch, which can be optionally generated from a text-to-image model like DALL-E 2. At generation time, our fine-tuned diffusion model is used through a score aggregation strategy to generate output texture images of arbitrary resolution on a single GPU. We compare synthesized textures from our method to existing work in patch-based and deep learning texture synthesis methods. We also showcase two applications of our generated textures in 3D rendering and texture transfer.