Pyramidal Predictive Network: A Model for Visual-frame Prediction Based on Predictive Coding Theory
This work addresses visual-frame prediction for computer vision applications, but it is incremental as it builds on existing predictive coding theory and aims for comparable rather than breakthrough performance.
The paper tackles the problems of lacking appearance details, low prediction accuracy, and high computational overhead in visual-frame prediction by proposing a novel neural network model based on predictive coding theory, which shows better compactness and comparable predictive performance with existing works, implying lower computational cost and higher prediction accuracy.
Visual-frame prediction is a pixel-dense prediction task that infers future frames from past frames. Lacking of appearance details, low prediction accuracy and high computational overhead are still major problems with current models or methods. In this paper, we propose a novel neural network model inspired by the well-known predictive coding theory to deal with the problems. Predictive coding provides an interesting and reliable computational framework, which will be combined with other theories such as the cerebral cortex at different level oscillates at different frequencies, to design an efficient and reliable predictive network model for visual-frame prediction. Specifically, the model is composed of a series of recurrent and convolutional units forming the top-down and bottom-up streams, respectively. The update frequency of neural units on each of the layer decreases with the increasing of network levels, which results in neurons of higher-level can capture information in longer time dimensions. According to the experimental results, this model shows better compactness and comparable predictive performance with existing works, implying lower computational cost and higher prediction accuracy. Code is available at https://github.com/Ling-CF/PPNet.