Exploiting Data Hierarchy as a New Modality for Contrastive Learning
This work addresses the challenge of weakly-supervised learning for domain-specific tasks like cathedral analysis, but it is incremental as it builds on existing contrastive learning methods.
The paper tackled the problem of learning conceptual representations of cathedrals by exploiting hierarchical data structure as a modality for contrastive learning, and the result showed that the proposed method outperformed comparable weakly-supervised and baseline methods in a downstream classification task.
This work investigates how hierarchically structured data can help neural networks learn conceptual representations of cathedrals. The underlying WikiScenes dataset provides a spatially organized hierarchical structure of cathedral components. We propose a novel hierarchical contrastive training approach that leverages a triplet margin loss to represent the data's spatial hierarchy in the encoder's latent space. As such, the proposed approach investigates if the dataset structure provides valuable information for self-supervised learning. We apply t-SNE to visualize the resultant latent space and evaluate the proposed approach by comparing it with other dataset-specific contrastive learning methods using a common downstream classification task. The proposed method outperforms the comparable weakly-supervised and baseline methods. Our findings suggest that dataset structure is a valuable modality for weakly-supervised learning.