GraphX-Convolution for Point Cloud Deformation in 2D-to-3D Conversion
This addresses the challenge of accurate and scalable 3D model reconstruction from images, which is incremental as it builds on prior deep methods but introduces novel capabilities.
The paper tackles the problem of reconstructing a 3D point cloud from a single 2D image by deforming a random point cloud through feature blending and a new GraphX layer, achieving a halving of the state-of-the-art distance score.
In this paper, we present a novel deep method to reconstruct a point cloud of an object from a single still image. Prior arts in the field struggle to reconstruct an accurate and scalable 3D model due to either the inefficient and expensive 3D representations, the dependency between the output and number of model parameters or the lack of a suitable computing operation. We propose to overcome these by deforming a random point cloud to the object shape through two steps: feature blending and deformation. In the first step, the global and point-specific shape features extracted from a 2D object image are blended with the encoded feature of a randomly generated point cloud, and then this mixture is sent to the deformation step to produce the final representative point set of the object. In the deformation process, we introduce a new layer termed as GraphX that considers the inter-relationship between points like common graph convolutions but operates on unordered sets. Moreover, with a simple trick, the proposed model can generate an arbitrary-sized point cloud, which is the first deep method to do so. Extensive experiments verify that we outperform existing models and halve the state-of-the-art distance score in single image 3D reconstruction.