SRVP: Strong Recollection Video Prediction Model Using Attention-Based Spatiotemporal Correlation Fusion
This work addresses a specific issue in video prediction for computer vision applications, but it is incremental as it builds on existing attention mechanisms to improve RNN-based models.
The paper tackles the problem of object appearance detail loss in recurrent neural network-based video prediction models by proposing the SRVP model, which integrates attention modules to enhance spatiotemporal representations, achieving predictive performance comparable to RNN-free architectures on three benchmark datasets.
Video prediction (VP) generates future frames by leveraging spatial representations and temporal context from past frames. Traditional recurrent neural network (RNN)-based models enhance memory cell structures to capture spatiotemporal states over extended durations but suffer from gradual loss of object appearance details. To address this issue, we propose the strong recollection VP (SRVP) model, which integrates standard attention (SA) and reinforced feature attention (RFA) modules. Both modules employ scaled dot-product attention to extract temporal context and spatial correlations, which are then fused to enhance spatiotemporal representations. Experiments on three benchmark datasets demonstrate that SRVP mitigates image quality degradation in RNN-based models while achieving predictive performance comparable to RNN-free architectures.