A Dynamic Multi-Scale Voxel Flow Network for Video Prediction
This work addresses efficiency and performance issues in video prediction for applications like robotics and autonomous driving, though it appears incremental as it builds on existing voxel flow and iterative-based methods.
The paper tackles video prediction by proposing a Dynamic Multi-scale Voxel Flow Network (DMVFN) that achieves better performance at lower computational costs using only RGB images, surpassing state-of-the-art methods in speed and image quality.
The performance of video prediction has been greatly boosted by advanced deep neural networks. However, most of the current methods suffer from large model sizes and require extra inputs, e.g., semantic/depth maps, for promising performance. For efficiency consideration, in this paper, we propose a Dynamic Multi-scale Voxel Flow Network (DMVFN) to achieve better video prediction performance at lower computational costs with only RGB images, than previous methods. The core of our DMVFN is a differentiable routing module that can effectively perceive the motion scales of video frames. Once trained, our DMVFN selects adaptive sub-networks for different inputs at the inference stage. Experiments on several benchmarks demonstrate that our DMVFN is an order of magnitude faster than Deep Voxel Flow and surpasses the state-of-the-art iterative-based OPT on generated image quality. Our code and demo are available at https://huxiaotaostasy.github.io/DMVFN/.