MetaSets: Meta-Learning on Point Sets for Generalizable Representations
This addresses the costly annotation of point clouds for 3D vision tasks by enabling better transfer across domains, though it is incremental as it builds on meta-learning for domain generalization.
The paper tackles the problem of 3D Domain Generalization for point clouds, where models must generalize to unseen domains without access during training, and proposes MetaSets, which meta-learns representations from transformed point sets, achieving large-margin improvements over existing methods.
Deep learning techniques for point clouds have achieved strong performance on a range of 3D vision tasks. However, it is costly to annotate large-scale point sets, making it critical to learn generalizable representations that can transfer well across different point sets. In this paper, we study a new problem of 3D Domain Generalization (3DDG) with the goal to generalize the model to other unseen domains of point clouds without any access to them in the training process. It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain. We propose to tackle this problem via MetaSets, which meta-learns point cloud representations from a group of classification tasks on carefully-designed transformed point sets containing specific geometry priors. The learned representations are more generalizable to various unseen domains of different geometries. We design two benchmarks for Sim-to-Real transfer of 3D point clouds. Experimental results show that MetaSets outperforms existing 3D deep learning methods by large margins.