TexPro: Text-guided PBR Texturing with Procedural Material Modeling
This addresses the need for efficient and realistic material generation in 3D modeling and rendering, offering a novel approach that goes beyond simple RGB textures.
The paper tackles the problem of generating high-fidelity, physically-based rendering (PBR) textures for 3D meshes from text prompts, achieving superior results over existing state-of-the-art methods and enabling capabilities like relighting.
In this paper, we present TexPro, a novel method for high-fidelity material generation for input 3D meshes given text prompts. Unlike existing text-conditioned texture generation methods that typically generate RGB textures with baked lighting, TexPro is able to produce diverse texture maps via procedural material modeling, which enables physically-based rendering, relighting, and additional benefits inherent to procedural materials. Specifically, we first generate multi-view reference images given the input textual prompt by employing the latest text-to-image model. We then derive texture maps through rendering-based optimization with recent differentiable procedural materials. To this end, we design several techniques to handle the misalignment between the generated multi-view images and 3D meshes, and introduce a novel material agent that enhances material classification and matching by exploring both part-level understanding and object-aware material reasoning. Experiments demonstrate the superiority of the proposed method over existing SOTAs, and its capability of relighting.