Object Wake-up: 3D Object Rigging from a Single Image
This work addresses the challenge of articulated 3D object rigging from single images, which is incremental as it builds on prior object manipulation efforts by extending to articulated manipulation.
The paper tackles the problem of reconstructing articulated 3D shapes from a single image of general objects like chairs, enabling animation for AR/VR applications, and reports results that surpass state-of-the-art methods by a noticeable margin on 3D reconstruction and skeleton prediction.
Given a single image of a general object such as a chair, could we also restore its articulated 3D shape similar to human modeling, so as to animate its plausible articulations and diverse motions? This is an interesting new question that may have numerous downstream augmented reality and virtual reality applications. Comparing with previous efforts on object manipulation, our work goes beyond 2D manipulation and rigid deformation, and involves articulated manipulation. To achieve this goal, we propose an automated approach to build such 3D generic objects from single images and embed articulated skeletons in them. Specifically, our framework starts by reconstructing the 3D object from an input image. Afterwards, to extract skeletons for generic 3D objects, we develop a novel skeleton prediction method with a multi-head structure for skeleton probability field estimation by utilizing the deep implicit functions. A dataset of generic 3D objects with ground-truth annotated skeletons is collected. Empirically our approach is demonstrated with satisfactory performance on public datasets as well as our in-house dataset; our results surpass those of the state-of-the-arts by a noticeable margin on both 3D reconstruction and skeleton prediction.