UV-free Texture Generation with Denoising and Geodesic Heat Diffusions
This addresses texture generation issues for 3D modeling and computer graphics, offering an incremental improvement over existing UV-based methods.
The paper tackles the problems of UV-based texturing for meshes, such as seams and distortions, by proposing a method that generates textures as colored point-clouds using a denoising diffusion model constrained to 3D surfaces, achieving resolution-agnostic results without UV unwrapping.
Seams, distortions, wasted UV space, vertex-duplication, and varying resolution over the surface are the most prominent issues of the standard UV-based texturing of meshes. These issues are particularly acute when automatic UV-unwrapping techniques are used. For this reason, instead of generating textures in automatically generated UV-planes like most state-of-the-art methods, we propose to represent textures as coloured point-clouds whose colours are generated by a denoising diffusion probabilistic model constrained to operate on the surface of 3D objects. Our sampling and resolution agnostic generative model heavily relies on heat diffusion over the surface of the meshes for spatial communication between points. To enable processing of arbitrarily sampled point-cloud textures and ensure long-distance texture consistency we introduce a fast re-sampling of the mesh spectral properties used during the heat diffusion and introduce a novel heat-diffusion-based self-attention mechanism. Our code and pre-trained models are available at github.com/simofoti/UV3-TeD.