One-to-many Reconstruction of 3D Geometry of cultural Artifacts using a synthetically trained Generative Model
This addresses the challenge of 3D reconstruction from less expressive inputs like historic sketches for cultural heritage applications, but it appears incremental as it adapts existing generative methods to a specific domain.
The paper tackles the problem of estimating 3D shapes from single images, specifically for cultural artifacts like medieval statues using sketches, by developing an automated approach that generates detailed 3D representations from a single sketch with multi-modal guidance and synthetic training data. The result is a system that enables domain experts to interactively reconstruct potential appearances of lost artifacts, though no concrete performance numbers are provided.
Estimating the 3D shape of an object using a single image is a difficult problem. Modern approaches achieve good results for general objects, based on real photographs, but worse results on less expressive representations such as historic sketches. Our automated approach generates a variety of detailed 3D representation from a single sketch, depicting a medieval statue, and can be guided by multi-modal inputs, such as text prompts. It relies solely on synthetic data for training, making it adoptable even in cases of only small numbers of training examples. Our solution allows domain experts such as a curators to interactively reconstruct potential appearances of lost artifacts.