3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation
This work addresses the challenge of local stylization for 3D shapes in computer graphics, offering a domain-specific solution for texture editing.
The paper tackles the problem of automatically texturing local semantic regions on 3D meshes using text descriptions, resulting in a method that produces high-quality texture maps and localization maps with improved detail and resolution through cascaded supervision.
In this work we develop 3D Paintbrush, a technique for automatically texturing local semantic regions on meshes via text descriptions. Our method is designed to operate directly on meshes, producing texture maps which seamlessly integrate into standard graphics pipelines. We opt to simultaneously produce a localization map (to specify the edit region) and a texture map which conforms to it. This synergistic approach improves the quality of both the localization and the stylization. To enhance the details and resolution of the textured area, we leverage multiple stages of a cascaded diffusion model to supervise our local editing technique with generative priors learned from images at different resolutions. Our technique, referred to as Cascaded Score Distillation (CSD), simultaneously distills scores at multiple resolutions in a cascaded fashion, enabling control over both the granularity and global understanding of the supervision. We demonstrate the effectiveness of 3D Paintbrush to locally texture a variety of shapes within different semantic regions. Project page: https://threedle.github.io/3d-paintbrush