Video Prediction via Example Guidance
This work addresses the problem of multi-modal video prediction for computer vision applications, offering an incremental improvement by integrating with existing stochastic models.
The paper tackles the challenge of capturing multi-modal future contents and dynamics in video prediction by proposing a framework that uses analogous expert examples from a training pool to approximate potential distributions, resulting in significant enhancement in both quantitative and qualitative aspects.
In video prediction tasks, one major challenge is to capture the multi-modal nature of future contents and dynamics. In this work, we propose a simple yet effective framework that can efficiently predict plausible future states. The key insight is that the potential distribution of a sequence could be approximated with analogous ones in a repertoire of training pool, namely, expert examples. By further incorporating a novel optimization scheme into the training procedure, plausible predictions can be sampled efficiently from distribution constructed from the retrieved examples. Meanwhile, our method could be seamlessly integrated with existing stochastic predictive models; significant enhancement is observed with comprehensive experiments in both quantitative and qualitative aspects. We also demonstrate the generalization ability to predict the motion of unseen class, i.e., without access to corresponding data during training phase.