GenesisTex: Adapting Image Denoising Diffusion to Texture Space
This addresses texture synthesis for 3D models, offering a more stable and efficient alternative to existing methods, though it appears incremental as it builds on diffusion models.
The paper tackles the problem of synthesizing textures for 3D geometries from text descriptions by adapting a pretrained image diffusion model to texture space, achieving results that surpass baseline methods quantitatively and qualitatively.
We present GenesisTex, a novel method for synthesizing textures for 3D geometries from text descriptions. GenesisTex adapts the pretrained image diffusion model to texture space by texture space sampling. Specifically, we maintain a latent texture map for each viewpoint, which is updated with predicted noise on the rendering of the corresponding viewpoint. The sampled latent texture maps are then decoded into a final texture map. During the sampling process, we focus on both global and local consistency across multiple viewpoints: global consistency is achieved through the integration of style consistency mechanisms within the noise prediction network, and low-level consistency is achieved by dynamically aligning latent textures. Finally, we apply reference-based inpainting and img2img on denser views for texture refinement. Our approach overcomes the limitations of slow optimization in distillation-based methods and instability in inpainting-based methods. Experiments on meshes from various sources demonstrate that our method surpasses the baseline methods quantitatively and qualitatively.