3D Shape Segmentation with Geometric Deep Learning
This addresses a computational bottleneck in 3D shape analysis for applications like computer graphics or robotics, but it appears incremental as it builds on existing deep learning and projection techniques.
The paper tackles the problem of semantic segmentation of high-density 3D shapes, which is computationally challenging due to large memory requirements, by proposing a neural-network approach that uses 3D augmented views and CNNs to achieve segmentation, validated on public datasets and real objects with comparisons to state-of-the-art methods.
The semantic segmentation of 3D shapes with a high-density of vertices could be impractical due to large memory requirements. To make this problem computationally tractable, we propose a neural-network based approach that produces 3D augmented views of the 3D shape to solve the whole segmentation as sub-segmentation problems. 3D augmented views are obtained by projecting vertices and normals of a 3D shape onto 2D regular grids taken from different viewpoints around the shape. These 3D views are then processed by a Convolutional Neural Network to produce a probability distribution function (pdf) over the set of the semantic classes for each vertex. These pdfs are then re-projected on the original 3D shape and postprocessed using contextual information through Conditional Random Fields. We validate our approach using 3D shapes of publicly available datasets and of real objects that are reconstructed using photogrammetry techniques. We compare our approach against state-of-the-art alternatives.