3D Shape Segmentation with Projective Convolutional Networks
This addresses the problem of accurately segmenting 3D objects into semantic parts for applications like computer vision and robotics, representing an incremental improvement over prior methods.
The paper tackles 3D shape segmentation by introducing a deep architecture that combines image-based Fully Convolutional Networks (FCNs) with surface-based Conditional Random Fields (CRFs) for end-to-end training, significantly outperforming existing state-of-the-art methods on the ShapeNet benchmark.
This paper introduces a deep architecture for segmenting 3D objects into their labeled semantic parts. Our architecture combines image-based Fully Convolutional Networks (FCNs) and surface-based Conditional Random Fields (CRFs) to yield coherent segmentations of 3D shapes. The image-based FCNs are used for efficient view-based reasoning about 3D object parts. Through a special projection layer, FCN outputs are effectively aggregated across multiple views and scales, then are projected onto the 3D object surfaces. Finally, a surface-based CRF combines the projected outputs with geometric consistency cues to yield coherent segmentations. The whole architecture (multi-view FCNs and CRF) is trained end-to-end. Our approach significantly outperforms the existing state-of-the-art methods in the currently largest segmentation benchmark (ShapeNet). Finally, we demonstrate promising segmentation results on noisy 3D shapes acquired from consumer-grade depth cameras.