S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation
This addresses representation learning for sequential data, offering a self-supervised approach that reduces reliance on labeled data, though it appears incremental as it builds on existing VAE and self-supervision techniques.
The authors tackled the problem of learning disentangled representations for sequential data like videos and audios using a self-supervised sequential VAE, achieving performance comparable to fully-supervised models and outperforming unsupervised state-of-the-art methods by a large margin.
We propose a sequential variational autoencoder to learn disentangled representations of sequential data (e.g., videos and audios) under self-supervision. Specifically, we exploit the benefits of some readily accessible supervisory signals from input data itself or some off-the-shelf functional models and accordingly design auxiliary tasks for our model to utilize these signals. With the supervision of the signals, our model can easily disentangle the representation of an input sequence into static factors and dynamic factors (i.e., time-invariant and time-varying parts). Comprehensive experiments across videos and audios verify the effectiveness of our model on representation disentanglement and generation of sequential data, and demonstrate that, our model with self-supervision performs comparable to, if not better than, the fully-supervised model with ground truth labels, and outperforms state-of-the-art unsupervised models by a large margin.