Hyperbolic Contrastive Learning
This work addresses the challenge of representing non-Euclidean data structures in computer vision, offering a novel approach for tasks like social networks and brain imaging, though it appears incremental as it builds on existing contrastive learning methods.
The authors tackled the problem of learning image representations for downstream tasks by proposing Hyperbolic Contrastive Learning (HCL), a framework that models hierarchical structures in hyperbolic space, achieving better results in self-supervised pretraining, supervised classification, and adversarial robustness compared to baseline methods.
Learning good image representations that are beneficial to downstream tasks is a challenging task in computer vision. As such, a wide variety of self-supervised learning approaches have been proposed. Among them, contrastive learning has shown competitive performance on several benchmark datasets. The embeddings of contrastive learning are arranged on a hypersphere that results in using the inner (dot) product as a distance measurement in Euclidean space. However, the underlying structure of many scientific fields like social networks, brain imaging, and computer graphics data exhibit highly non-Euclidean latent geometry. We propose a novel contrastive learning framework to learn semantic relationships in the hyperbolic space. Hyperbolic space is a continuous version of trees that naturally owns the ability to model hierarchical structures and is thus beneficial for efficient contrastive representation learning. We also extend the proposed Hyperbolic Contrastive Learning (HCL) to the supervised domain and studied the adversarial robustness of HCL. The comprehensive experiments show that our proposed method achieves better results on self-supervised pretraining, supervised classification, and higher robust accuracy than baseline methods.