Anti-aliasing Predictive Coding Network for Future Video Frame Prediction
This work addresses video frame prediction for applications like autonomous driving or video compression, but it is incremental, building on existing predictive coding methods.
The paper tackles the problem of generating sharp and accurate future video frames by introducing a predictive coding network with modulated feature fusion and redesigned downsampling/upsampling modules, achieving a better balance between pixel accuracy and visual quality.
We introduce here a predictive coding based model that aims to generate accurate and sharp future frames. Inspired by the predictive coding hypothesis and related works, the total model is updated through a combination of bottom-up and top-down information flows, which can enhance the interaction between different network levels. Most importantly, We propose and improve several artifacts to ensure that the neural networks generate clear and natural frames. Different inputs are no longer simply concatenated or added, they are calculated in a modulated manner to avoid being roughly fused. The downsampling and upsampling modules have been redesigned to ensure that the network can more easily construct images from Fourier features of low-frequency inputs. Additionally, the training strategies are also explored and improved to generate believable results and alleviate inconsistency between the input predicted frames and ground truth. Our proposals achieve results that better balance pixel accuracy and visualization effect.