Continual Learning of Predictive Models in Video Sequences via Variational Autoencoders
This addresses the problem of catastrophic forgetting in video prediction for autonomous systems, but it is incremental as it builds on existing methods like VAEs and particle filters.
The paper tackles continual learning of predictive models for future frame inference in video sequences, achieving integration of new situations without catastrophic forgetting in a controlled vehicle environment.
This paper proposes a method for performing continual learning of predictive models that facilitate the inference of future frames in video sequences. For a first given experience, an initial Variational Autoencoder, together with a set of fully connected neural networks are utilized to respectively learn the appearance of video frames and their dynamics at the latent space level. By employing an adapted Markov Jump Particle Filter, the proposed method recognizes new situations and integrates them as predictive models avoiding catastrophic forgetting of previously learned tasks. For evaluating the proposed method, this article uses video sequences from a vehicle that performs different tasks in a controlled environment.