A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
This work addresses unsupervised learning for video analysis, enabling imagination and imputation without generating frames, but it is incremental as it builds on existing variational auto-encoder frameworks.
The paper tackled the problem of temporal reasoning in videos by disentangling object representations from their nonlinear dynamics in latent space, resulting in a model that outperformed competing methods in generative and missing data imputation tasks on simulated physical systems.
This paper takes a step towards temporal reasoning in a dynamically changing video, not in the pixel space that constitutes its frames, but in a latent space that describes the non-linear dynamics of the objects in its world. We introduce the Kalman variational auto-encoder, a framework for unsupervised learning of sequential data that disentangles two latent representations: an object's representation, coming from a recognition model, and a latent state describing its dynamics. As a result, the evolution of the world can be imagined and missing data imputed, both without the need to generate high dimensional frames at each time step. The model is trained end-to-end on videos of a variety of simulated physical systems, and outperforms competing methods in generative and missing data imputation tasks.