WIR3D: Visually-Informed and Geometry-Aware 3D Shape Abstraction
This work addresses the need for intuitive 3D shape abstraction and control for users in computer graphics and design, though it is incremental as it builds on existing foundation models and optimization techniques.
The paper tackles the problem of abstracting 3D shapes into sparse, visually meaningful curves by optimizing Bezier parameters to represent geometry and texture from arbitrary viewpoints, achieving successful application across diverse shapes with varying complexity and texture.
In this work we present WIR3D, a technique for abstracting 3D shapes through a sparse set of visually meaningful curves in 3D. We optimize the parameters of Bezier curves such that they faithfully represent both the geometry and salient visual features (e.g. texture) of the shape from arbitrary viewpoints. We leverage the intermediate activations of a pre-trained foundation model (CLIP) to guide our optimization process. We divide our optimization into two phases: one for capturing the coarse geometry of the shape, and the other for representing fine-grained features. Our second phase supervision is spatially guided by a novel localized keypoint loss. This spatial guidance enables user control over abstracted features. We ensure fidelity to the original surface through a neural SDF loss, which allows the curves to be used as intuitive deformation handles. We successfully apply our method for shape abstraction over a broad dataset of shapes with varying complexity, geometric structure, and texture, and demonstrate downstream applications for feature control and shape deformation.