S-JEA: Stacked Joint Embedding Architectures for Self-Supervised Visual Representation Learning
This work addresses the need for more interpretable and structured embeddings in self-supervised learning for computer vision, but it is incremental as it builds on existing joint embedding architectures without achieving significant performance gains.
The paper tackled the problem of learning hierarchical and separable semantic concepts in self-supervised visual representation learning by stacking joint embedding architectures, resulting in representations that perform similarly to traditional methods with comparable parameters.
The recent emergence of Self-Supervised Learning (SSL) as a fundamental paradigm for learning image representations has, and continues to, demonstrate high empirical success in a variety of tasks. However, most SSL approaches fail to learn embeddings that capture hierarchical semantic concepts that are separable and interpretable. In this work, we aim to learn highly separable semantic hierarchical representations by stacking Joint Embedding Architectures (JEA) where higher-level JEAs are input with representations of lower-level JEA. This results in a representation space that exhibits distinct sub-categories of semantic concepts (e.g., model and colour of vehicles) in higher-level JEAs. We empirically show that representations from stacked JEA perform on a similar level as traditional JEA with comparative parameter counts and visualise the representation spaces to validate the semantic hierarchies.