Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactions
This addresses the problem of accurate video forecasting for applications like robotics or autonomous systems, but it is incremental as it builds on existing object-centric methods with a decoupling approach.
The paper tackles object-centric video prediction by extracting compositional structure and modeling object dynamics and interactions to forecast future states and generate frames, resulting in improved performance over object-agnostic models on two datasets.
We propose a novel framework for the task of object-centric video prediction, i.e., extracting the compositional structure of a video sequence, as well as modeling objects dynamics and interactions from visual observations in order to predict the future object states, from which we can then generate subsequent video frames. With the goal of learning meaningful spatio-temporal object representations and accurately forecasting object states, we propose two novel object-centric video predictor (OCVP) transformer modules, which decouple the processing of temporal dynamics and object interactions, thus presenting an improved prediction performance. In our experiments, we show how our object-centric prediction framework utilizing our OCVP predictors outperforms object-agnostic video prediction models on two different datasets, while maintaining consistent and accurate object representations.