MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignment
This addresses the challenge of generating high-quality, consistent textures and materials for 3D models from text descriptions, enabling more editable and relightable assets in computer graphics and AI applications.
The authors tackled the problem of consistent text-guided 3D model texturing by developing MatAtlas, which uses a text-to-image model and a multi-step refinement process to improve quality and 3D consistency, and they introduced a material retrieval method using LLMs for editability and relightability, showing significant outperformance over prior methods.
We present MatAtlas, a method for consistent text-guided 3D model texturing. Following recent progress we leverage a large scale text-to-image generation model (e.g., Stable Diffusion) as a prior to texture a 3D model. We carefully design an RGB texturing pipeline that leverages a grid pattern diffusion, driven by depth and edges. By proposing a multi-step texture refinement process, we significantly improve the quality and 3D consistency of the texturing output. To further address the problem of baked-in lighting, we move beyond RGB colors and pursue assigning parametric materials to the assets. Given the high-quality initial RGB texture, we propose a novel material retrieval method capitalized on Large Language Models (LLM), enabling editabiliy and relightability. We evaluate our method on a wide variety of geometries and show that our method significantly outperform prior arts. We also analyze the role of each component through a detailed ablation study.