Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches
This addresses the long-standing open problem of 3D shape reconstruction from sketches for applications in design and modeling, though it appears incremental as it builds on existing encoder/decoder architectures.
The paper tackles the problem of reconstructing 3D shapes from 2D sketches, which is challenging due to sparse and ambiguous information, by using an encoder/decoder architecture that integrates camera parameters to match sketch contours. It demonstrates that this approach outperforms state-of-the-art methods on both hand-drawn and synthesized sketches.
Reconstructing 3D shape from 2D sketches has long been an open problem because the sketches only provide very sparse and ambiguous information. In this paper, we use an encoder/decoder architecture for the sketch to mesh translation. When integrated into a user interface that provides camera parameters for the sketches, this enables us to leverage its latent parametrization to represent and refine a 3D mesh so that its projections match the external contours outlined in the sketch. We will show that this approach is easy to deploy, robust to style changes, and effective. Furthermore, it can be used for shape refinement given only single pen strokes. We compare our approach to state-of-the-art methods on sketches -- both hand-drawn and synthesized -- and demonstrate that we outperform them.