FG-DFPN: Flow Guided Deformable Frame Prediction Network
This addresses the problem of high-fidelity temporal predictions for applications like autonomous systems and video compression, representing a strong specific gain in the domain.
The paper tackles video frame prediction by proposing FG-DFPN, which integrates optical flow and deformable convolutions to model spatio-temporal dynamics, achieving state-of-the-art performance with a 1dB PSNR improvement on eight MPEG test sequences.
Video frame prediction remains a fundamental challenge in computer vision with direct implications for autonomous systems, video compression, and media synthesis. We present FG-DFPN, a novel architecture that harnesses the synergy between optical flow estimation and deformable convolutions to model complex spatio-temporal dynamics. By guiding deformable sampling with motion cues, our approach addresses the limitations of fixed-kernel networks when handling diverse motion patterns. The multi-scale design enables FG-DFPN to simultaneously capture global scene transformations and local object movements with remarkable precision. Our experiments demonstrate that FG-DFPN achieves state-of-the-art performance on eight diverse MPEG test sequences, outperforming existing methods by 1dB PSNR while maintaining competitive inference speeds. The integration of motion cues with adaptive geometric transformations makes FG-DFPN a promising solution for next-generation video processing systems that require high-fidelity temporal predictions. The model and instructions to reproduce our results will be released at: https://github.com/KUIS-AI-Tekalp-Research Group/frame-prediction