Point Convolutional Neural Networks by Extension Operators
This addresses the challenge of efficient and robust deep learning on point clouds for applications like 3D vision, with a novel approach that is not incremental.
The paper tackles the problem of applying convolutional neural networks to point clouds by introducing a framework with extension and restriction operators, achieving state-of-the-art performance on three benchmarks and outperforming most methods using more complex shape representations.
This paper presents Point Convolutional Neural Networks (PCNN): a novel framework for applying convolutional neural networks to point clouds. The framework consists of two operators: extension and restriction, mapping point cloud functions to volumetric functions and vise-versa. A point cloud convolution is defined by pull-back of the Euclidean volumetric convolution via an extension-restriction mechanism. The point cloud convolution is computationally efficient, invariant to the order of points in the point cloud, robust to different samplings and varying densities, and translation invariant, that is the same convolution kernel is used at all points. PCNN generalizes image CNNs and allows readily adapting their architectures to the point cloud setting. Evaluation of PCNN on three central point cloud learning benchmarks convincingly outperform competing point cloud learning methods, and the vast majority of methods working with more informative shape representations such as surfaces and/or normals.