Image-Guided Geometric Stylization of 3D Meshes
This addresses the need for artistic 3D content creation by enabling image-guided geometric stylization, though it is incremental as it builds on existing generative models.
The paper tackles the problem of generating 3D meshes with bold geometric distortions based on image styles, using a coarse-to-fine pipeline that deforms meshes to reflect image features while preserving topology and semantics, resulting in stylized 3D creations with expressive poses and silhouettes.
Recent generative models can create visually plausible 3D representations of objects. However, the generation process often allows for implicit control signals, such as contextual descriptions, and rarely supports bold geometric distortions beyond existing data distributions. We propose a geometric stylization framework that deforms a 3D mesh, allowing it to express the style of an image. While style is inherently ambiguous, we utilize pre-trained diffusion models to extract an abstract representation of the provided image. Our coarse-to-fine stylization pipeline can drastically deform the input 3D model to express a diverse range of geometric variations while retaining the valid topology of the original mesh and part-level semantics. We also propose an approximate VAE encoder that provides efficient and reliable gradients from mesh renderings. Extensive experiments demonstrate that our method can create stylized 3D meshes that reflect unique geometric features of the pictured assets, such as expressive poses and silhouettes, thereby supporting the creation of distinctive artistic 3D creations. Project page: https://changwoonchoi.github.io/GeoStyle