Cloud Transformers: A Universal Approach To Point Cloud Processing Tasks
This provides a universal solution for point cloud processing tasks, benefiting researchers and practitioners in computer vision and robotics, though it is incremental as it builds on existing ideas.
The authors tackled the problem of versatile point cloud processing by introducing a new building block that combines spatial transformers, multi-view convolutional networks, and dense convolutions, achieving state-of-the-art performance in tasks like segmentation, classification, inpainting, and reconstruction.
We present a new versatile building block for deep point cloud processing architectures that is equally suited for diverse tasks. This building block combines the ideas of spatial transformers and multi-view convolutional networks with the efficiency of standard convolutional layers in two and three-dimensional dense grids. The new block operates via multiple parallel heads, whereas each head differentiably rasterizes feature representations of individual points into a low-dimensional space, and then uses dense convolution to propagate information across points. The results of the processing of individual heads are then combined together resulting in the update of point features. Using the new block, we build architectures for both discriminative (point cloud segmentation, point cloud classification) and generative (point cloud inpainting and image-based point cloud reconstruction) tasks. The resulting architectures achieve state-of-the-art performance for these tasks, demonstrating the versatility of the new block for point cloud processing.