3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks
This addresses the need for compact and abstract 3D representations in robotics and digital content creation, but it is incremental as it builds on existing generative models with a focus on primitive-based shapes.
The paper tackles the problem of generating structured 3D shape representations from single depth images by proposing 3D-PRNN, a generative recurrent neural network that synthesizes shapes composed of primitives, outperforming nearest-neighbor methods and matching voxel-based models with reduced parameters.
The success of various applications including robotics, digital content creation, and visualization demand a structured and abstract representation of the 3D world from limited sensor data. Inspired by the nature of human perception of 3D shapes as a collection of simple parts, we explore such an abstract shape representation based on primitives. Given a single depth image of an object, we present 3D-PRNN, a generative recurrent neural network that synthesizes multiple plausible shapes composed of a set of primitives. Our generative model encodes symmetry characteristics of common man-made objects, preserves long-range structural coherence, and describes objects of varying complexity with a compact representation. We also propose a method based on Gaussian Fields to generate a large scale dataset of primitive-based shape representations to train our network. We evaluate our approach on a wide range of examples and show that it outperforms nearest-neighbor based shape retrieval methods and is on-par with voxel-based generative models while using a significantly reduced parameter space.