ParaNet: Deep Regular Representation for 3D Point Clouds
This work addresses the challenge of applying 2D deep learning techniques to irregular 3D point cloud data, which is a problem for researchers and practitioners working with 3D data.
This paper introduces ParaNet, a deep learning framework that transforms irregular 3D point clouds into regular 2D color images called Point Geometry Images (PGIs). This representation is differentiable and reversible, enabling the application of standard 2D CNNs to 3D point cloud tasks. ParaNet achieves favorable performance against state-of-the-art methods in shape classification and point cloud upsampling.
Although convolutional neural networks have achieved remarkable success in analyzing 2D images/videos, it is still non-trivial to apply the well-developed 2D techniques in regular domains to the irregular 3D point cloud data. To bridge this gap, we propose ParaNet, a novel end-to-end deep learning framework, for representing 3D point clouds in a completely regular and nearly lossless manner. To be specific, ParaNet converts an irregular 3D point cloud into a regular 2D color image, named point geometry image (PGI), where each pixel encodes the spatial coordinates of a point. In contrast to conventional regular representation modalities based on multi-view projection and voxelization, the proposed representation is differentiable and reversible. Technically, ParaNet is composed of a surface embedding module, which parameterizes 3D surface points onto a unit square, and a grid resampling module, which resamples the embedded 2D manifold over regular dense grids. Note that ParaNet is unsupervised, i.e., the training simply relies on reference-free geometry constraints. The PGIs can be seamlessly coupled with a task network established upon standard and mature techniques for 2D images/videos to realize a specific task for 3D point clouds. We evaluate ParaNet over shape classification and point cloud upsampling, in which our solutions perform favorably against the existing state-of-the-art methods. We believe such a paradigm will open up many possibilities to advance the progress of deep learning-based point cloud processing and understanding.