Task-Aware Sampling Layer for Point-Wise Analysis
This work addresses a domain-specific bottleneck in point cloud processing for computer vision applications, offering an incremental improvement over existing sampling methods.
The paper tackles the problem of suboptimal uniform sampling in point cloud analysis by proposing a task-aware sampling layer that learns sampling point displacements supervised by task-related ground truth, resulting in improved performance over Farthest Point Sampling in tasks like semantic part segmentation, point cloud completion, and keypoint detection.
Sampling, grouping, and aggregation are three important components in the multi-scale analysis of point clouds. In this paper, we present a novel data-driven sampler learning strategy for point-wise analysis tasks. Unlike the widely used sampling technique, Farthest Point Sampling (FPS), we propose to learn sampling and downstream applications jointly. Our key insight is that uniform sampling methods like FPS are not always optimal for different tasks: sampling more points around boundary areas can make the point-wise classification easier for segmentation. Towards this end, we propose a novel sampler learning strategy that learns sampling point displacement supervised by task-related ground truth information and can be trained jointly with the underlying tasks. We further demonstrate our methods in various point-wise analysis tasks, including semantic part segmentation, point cloud completion, and keypoint detection. Our experiments show that jointly learning of the sampler and task brings better performance than using FPS in various point-based networks.