GenVDM: Generating Vector Displacement Maps From a Single Image
This work addresses a domain-specific problem for 3D artists by providing a novel tool to add and customize geometric details on 3D shapes, though it is incremental in focusing on parts rather than complete shapes.
The paper tackles the problem of generating Vector Displacement Maps (VDMs) from a single image, enabling the creation of detailed geometric stamps for 3D modeling, and demonstrates that their method outperforms existing baselines.
We introduce the first method for generating Vector Displacement Maps (VDMs): parameterized, detailed geometric stamps commonly used in 3D modeling. Given a single input image, our method first generates multi-view normal maps and then reconstructs a VDM from the normals via a novel reconstruction pipeline. We also propose an efficient algorithm for extracting VDMs from 3D objects, and present the first academic VDM dataset. Compared to existing 3D generative models focusing on complete shapes, we focus on generating parts that can be seamlessly attached to shape surfaces. The method gives artists rich control over adding geometric details to a 3D shape. Experiments demonstrate that our approach outperforms existing baselines. Generating VDMs offers additional benefits, such as using 2D image editing to customize and refine 3D details.