Nuvo: Neural UV Mapping for Unruly 3D Representations
This addresses a bottleneck in 3D reconstruction and generation for applications such as view synthesis and appearance editing, though it is incremental as it builds on existing neural field techniques.
The paper tackles the problem of UV mapping for 3D geometry from neural reconstruction techniques, which causes fragmented textures, and presents Nuvo, a neural field-based method that produces valid and well-behaved UV mappings for visible points, resulting in editable textures for tasks like view synthesis.
Existing UV mapping algorithms are designed to operate on well-behaved meshes, instead of the geometry representations produced by state-of-the-art 3D reconstruction and generation techniques. As such, applying these methods to the volume densities recovered by neural radiance fields and related techniques (or meshes triangulated from such fields) results in texture atlases that are too fragmented to be useful for tasks such as view synthesis or appearance editing. We present a UV mapping method designed to operate on geometry produced by 3D reconstruction and generation techniques. Instead of computing a mapping defined on a mesh's vertices, our method Nuvo uses a neural field to represent a continuous UV mapping, and optimizes it to be a valid and well-behaved mapping for just the set of visible points, i.e. only points that affect the scene's appearance. We show that our model is robust to the challenges posed by ill-behaved geometry, and that it produces editable UV mappings that can represent detailed appearance.