Transformation-Based Models of Video Sequences
This work addresses video frame prediction for computer vision applications, offering an incremental improvement in efficiency and evaluation.
The authors tackled next frame prediction in video by proposing an unsupervised method that predicts transformations between frames instead of pixels, leading to sharper results and a smaller model. They also introduced a new evaluation protocol using classifier performance, and their approach outperformed more sophisticated methods on UCF-101 while being more efficient.
In this work we propose a simple unsupervised approach for next frame prediction in video. Instead of directly predicting the pixels in a frame given past frames, we predict the transformations needed for generating the next frame in a sequence, given the transformations of the past frames. This leads to sharper results, while using a smaller prediction model. In order to enable a fair comparison between different video frame prediction models, we also propose a new evaluation protocol. We use generated frames as input to a classifier trained with ground truth sequences. This criterion guarantees that models scoring high are those producing sequences which preserve discriminative features, as opposed to merely penalizing any deviation, plausible or not, from the ground truth. Our proposed approach compares favourably against more sophisticated ones on the UCF-101 data set, while also being more efficient in terms of the number of parameters and computational cost.