Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimation
This addresses the challenge of disentangling representations in sequential data without external signals, offering a novel approach that could benefit fields like video and audio analysis, though it appears incremental relative to existing self-supervised techniques.
The paper tackles unsupervised sequential disentanglement by proposing a self-supervised framework that avoids modality-based augmentations, instead using contrastive estimation with latent space sampling. It achieves state-of-the-art results on video, audio, and time series benchmarks.
Unsupervised disentanglement is a long-standing challenge in representation learning. Recently, self-supervised techniques achieved impressive results in the sequential setting, where data is time-dependent. However, the latter methods employ modality-based data augmentations and random sampling or solve auxiliary tasks. In this work, we propose to avoid that by generating, sampling, and comparing empirical distributions from the underlying variational model. Unlike existing work, we introduce a self-supervised sequential disentanglement framework based on contrastive estimation with no external signals, while using common batch sizes and samples from the latent space itself. In practice, we propose a unified, efficient, and easy-to-code sampling strategy for semantically similar and dissimilar views of the data. We evaluate our approach on video, audio, and time series benchmarks. Our method presents state-of-the-art results in comparison to existing techniques. The code is available at https://github.com/azencot-group/SPYL.