PFCNN: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames
This addresses a problem in computer vision and graphics for researchers and practitioners working with 3D geometric data, offering a novel method for surface-based CNNs.
The authors tackled the challenge of applying convolutional neural networks directly to 3D surface meshes, which have non-Euclidean structures, by developing PFCNNs that use parallel frames to enable effective feature learning and achieve robust and superior performance in tasks like classification, segmentation, and registration.
Surface meshes are widely used shape representations and capture finer geometry data than point clouds or volumetric grids, but are challenging to apply CNNs directly due to their non-Euclidean structure. We use parallel frames on surface to define PFCNNs that enable effective feature learning on surface meshes by mimicking standard convolutions faithfully. In particular, the convolution of PFCNN not only maps local surface patches onto flat tangent planes, but also aligns the tangent planes such that they locally form a flat Euclidean structure, thus enabling recovery of standard convolutions. The alignment is achieved by the tool of locally flat connections borrowed from discrete differential geometry, which can be efficiently encoded and computed by parallel frame fields. In addition, the lack of canonical axis on surface is handled by sampling with the frame directions. Experiments show that for tasks including classification, segmentation and registration on deformable geometric domains, as well as semantic scene segmentation on rigid domains, PFCNNs achieve robust and superior performances without using sophisticated input features than state-of-the-art surface based CNNs.