Video Extrapolation with an Invertible Linear Embedding
This addresses video extrapolation for applications like forecasting and simulation, but it appears incremental as it builds on existing invertible neural network and dynamic system approaches.
The paper tackles video frame prediction from complex dynamic scenes by using an invertible neural network as an encoder for a nonlinear dynamic system with linear latent state evolution, demonstrating successful learning, prediction, and latent state inference without explicit reconstruction loss or simplistic pixel-space assumptions. It shows viability compared to a state-of-the-art method.
We predict future video frames from complex dynamic scenes, using an invertible neural network as the encoder of a nonlinear dynamic system with latent linear state evolution. Our invertible linear embedding (ILE) demonstrates successful learning, prediction and latent state inference. In contrast to other approaches, ILE does not use any explicit reconstruction loss or simplistic pixel-space assumptions. Instead, it leverages invertibility to optimize the likelihood of image sequences exactly, albeit indirectly. Comparison with a state-of-the-art method demonstrates the viability of our approach.