Towards a Unified Framework of Contrastive Learning for Disentangled Representations
This work addresses the problem of learning disentangled representations in machine learning, providing a more general theoretical framework, but it is incremental as it builds on existing contrastive learning approaches.
The paper extends theoretical guarantees for disentangling data representations to a broader family of contrastive learning methods, proving identifiability of true latents for four losses without common independence assumptions, and validates this on benchmark datasets.
Contrastive learning has recently emerged as a promising approach for learning data representations that discover and disentangle the explanatory factors of the data. Previous analyses of such approaches have largely focused on individual contrastive losses, such as noise-contrastive estimation (NCE) and InfoNCE, and rely on specific assumptions about the data generating process. This paper extends the theoretical guarantees for disentanglement to a broader family of contrastive methods, while also relaxing the assumptions about the data distribution. Specifically, we prove identifiability of the true latents for four contrastive losses studied in this paper, without imposing common independence assumptions. The theoretical findings are validated on several benchmark datasets. Finally, practical limitations of these methods are also investigated.