Hyperbolic Image-and-Pointcloud Contrastive Learning for 3D Classification
This work addresses 3D classification tasks, particularly for object and few-shot classification, by introducing a novel hyperbolic approach to enhance multi-modal contrastive learning.
The paper tackled the problem of insufficient exploration of hierarchical and semantic correlations in 3D contrastive learning by proposing HyperIPC, a hyperbolic image-and-pointcloud contrastive learning method, which improved object classification by 2.8% and few-shot classification by 5.9% on ScanObjectNN.
3D contrastive representation learning has exhibited remarkable efficacy across various downstream tasks. However, existing contrastive learning paradigms based on cosine similarity fail to deeply explore the potential intra-modal hierarchical and cross-modal semantic correlations about multi-modal data in Euclidean space. In response, we seek solutions in hyperbolic space and propose a hyperbolic image-and-pointcloud contrastive learning method (HyperIPC). For the intra-modal branch, we rely on the intrinsic geometric structure to explore the hyperbolic embedding representation of point cloud to capture invariant features. For the cross-modal branch, we leverage images to guide the point cloud in establishing strong semantic hierarchical correlations. Empirical experiments underscore the outstanding classification performance of HyperIPC. Notably, HyperIPC enhances object classification results by 2.8% and few-shot classification outcomes by 5.9% on ScanObjectNN compared to the baseline. Furthermore, ablation studies and confirmatory testing validate the rationality of HyperIPC's parameter settings and the effectiveness of its submodules.