Frequency Domain Transformer Networks for Video Prediction
This work addresses the challenge of video prediction for computer vision applications, but it appears incremental as it builds on existing methods with a domain-specific modification.
The paper tackles video prediction by proposing a Frequency Domain Transformer Network (FDTN) that operates in the frequency domain to address spatial nonlinearity, and it outperforms methods like Video Ladder Network and Predictive Gated Pyramids in experiments.
The task of video prediction is forecasting the next frames given some previous frames. Despite much recent progress, this task is still challenging mainly due to high nonlinearity in the spatial domain. To address this issue, we propose a novel architecture, Frequency Domain Transformer Network (FDTN), which is an end-to-end learnable model that estimates and uses the transformations of the signal in the frequency domain. Experimental evaluations show that this approach can outperform some widely used video prediction methods like Video Ladder Network (VLN) and Predictive Gated Pyramids (PGP).