Unsupervised Separation of Dynamics from Pixels
This addresses the challenge of unsupervised dynamic modeling in computer vision, though it appears incremental as it builds on existing probabilistic and state-space methods.
The paper tackles the problem of learning the dynamics of multiple objects from image sequences without supervision, achieving this by introducing a probabilistic model that separates linear dynamics from non-linear rendering, enabling efficient inference without retraining.
We present an approach to learn the dynamics of multiple objects from image sequences in an unsupervised way. We introduce a probabilistic model that first generate noisy positions for each object through a separate linear state-space model, and then renders the positions of all objects in the same image through a highly non-linear process. Such a linear representation of the dynamics enables us to propose an inference method that uses exact and efficient inference tools and that can be deployed to query the model in different ways without retraining.