SIAM: A Simple Alternating Mixer for Video Prediction
This addresses the need for a more versatile video prediction solution for applications like autonomous driving and weather forecasting, though it appears incremental as it builds on existing encoder-decoder approaches.
The paper tackled the problem of generic video prediction by proposing a simple alternating mixer (SIAM) that models spatial, temporal, and spatiotemporal features in a unified framework, achieving superior performance on four benchmark datasets.
Video prediction, predicting future frames from the previous ones, has broad applications such as autonomous driving and weather forecasting. Existing state-of-the-art methods typically focus on extracting either spatial, temporal, or spatiotemporal features from videos. Different feature focuses, resulting from different network architectures, may make the resultant models excel at some video prediction tasks but perform poorly on others. Towards a more generic video prediction solution, we explicitly model these features in a unified encoder-decoder framework and propose a novel simple alternating Mixer (SIAM). The novelty of SIAM lies in the design of dimension alternating mixing (DaMi) blocks, which can model spatial, temporal, and spatiotemporal features through alternating the dimensions of the feature maps. Extensive experimental results demonstrate the superior performance of the proposed SIAM on four benchmark video datasets covering both synthetic and real-world scenarios.