Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes
This work democratizes 3D content creation for users with limited drawing skills, though it appears incremental as it builds on existing sketch-to-3D methods with a novel part-level approach.
The paper tackles the problem of generating precise 3D shapes from abstract freehand sketches, overcoming limitations tied to drawing skills, and achieves this through a part-level modelling and alignment framework that eliminates the need for paired sketch-3D datasets and reduces computational demands.
In this paper, we democratise 3D content creation, enabling precise generation of 3D shapes from abstract sketches while overcoming limitations tied to drawing skills. We introduce a novel part-level modelling and alignment framework that facilitates abstraction modelling and cross-modal correspondence. Leveraging the same part-level decoder, our approach seamlessly extends to sketch modelling by establishing correspondence between CLIPasso edgemaps and projected 3D part regions, eliminating the need for a dataset pairing human sketches and 3D shapes. Additionally, our method introduces a seamless in-position editing process as a byproduct of cross-modal part-aligned modelling. Operating in a low-dimensional implicit space, our approach significantly reduces computational demands and processing time.