DECOLLAGE: 3D Detailization by Controllable, Localized, and Learned Geometry Enhancement
This work addresses the need for controllable and localized AI-assisted 3D content creation, offering an incremental improvement over existing detailization methods.
The paper tackles the problem of enabling end-users to refine 3D shapes by painting desired geometric details onto coarse voxel shapes, resulting in a method that generates high-resolution stylized geometries with more coherent details and style transitions compared to prior global techniques.
We present a 3D modeling method which enables end-users to refine or detailize 3D shapes using machine learning, expanding the capabilities of AI-assisted 3D content creation. Given a coarse voxel shape (e.g., one produced with a simple box extrusion tool or via generative modeling), a user can directly "paint" desired target styles representing compelling geometric details, from input exemplar shapes, over different regions of the coarse shape. These regions are then up-sampled into high-resolution geometries which adhere with the painted styles. To achieve such controllable and localized 3D detailization, we build on top of a Pyramid GAN by making it masking-aware. We devise novel structural losses and priors to ensure that our method preserves both desired coarse structures and fine-grained features even if the painted styles are borrowed from diverse sources, e.g., different semantic parts and even different shape categories. Through extensive experiments, we show that our ability to localize details enables novel interactive creative workflows and applications. Our experiments further demonstrate that in comparison to prior techniques built on global detailization, our method generates structure-preserving, high-resolution stylized geometries with more coherent shape details and style transitions.