SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation
This work addresses a specific issue in point cloud segmentation for computer vision applications, representing an incremental improvement over existing methods.
The paper tackles the problem of confusion in point cloud semantic segmentation due to class-agnostic local geometric features by proposing SemAffiNet, which enhances mid-level features with semantic information using semantic-affine transformation, achieving superior results on ScanNetV2 and NYUv2 datasets.
Conventional point cloud semantic segmentation methods usually employ an encoder-decoder architecture, where mid-level features are locally aggregated to extract geometric information. However, the over-reliance on these class-agnostic local geometric representations may raise confusion between local parts from different categories that are similar in appearance or spatially adjacent. To address this issue, we argue that mid-level features can be further enhanced with semantic information, and propose semantic-affine transformation that transforms features of mid-level points belonging to different categories with class-specific affine parameters. Based on this technique, we propose SemAffiNet for point cloud semantic segmentation, which utilizes the attention mechanism in the Transformer module to implicitly and explicitly capture global structural knowledge within local parts for overall comprehension of each category. We conduct extensive experiments on the ScanNetV2 and NYUv2 datasets, and evaluate semantic-affine transformation on various 3D point cloud and 2D image segmentation baselines, where both qualitative and quantitative results demonstrate the superiority and generalization ability of our proposed approach. Code is available at https://github.com/wangzy22/SemAffiNet.