Boosting Neural Video Codec via Scale-Driven Online Flow Refinement
It addresses the generalization problem of neural video codecs on unseen complex motion patterns without costly online fine-tuning, offering a practical improvement for video compression.
The paper proposes a training-free, plug-and-play module (SOFR) that refines motion estimation in neural video codecs by fusing coarse and fine-scale flow information, achieving 2.84% and 4.05% average bitrate savings in PSNR and MS-SSIM on DCVC-FM with negligible overhead.
Although state-of-the-art neural video codecs (NVCs) have achieved remarkable performance, they suffer from limited generalization when encountering complex motion patterns unseen during training. To bridge this domain gap without the expensive cost of online fine-tuning, we propose a Training-Free Scale-Driven Online Flow Refinement (SOFR) method. Serving as a plug-and-play module, SOFR integrates motion information from coarse and fine scales and dynamically fuses them according to warping accuracy, effectively rectifying motion estimation errors with negligible computational overhead. Furthermore, we design a rate-aware strategy that selects different dynamic fusion strategies according to bitrate modes, and employs a reliability check based on warping error to ensure robustness. Extensive experiments on the USTC-TD dataset verify the effectiveness and generalization of SOFR across various NVC frameworks, including DCVC-SDD, DCVC-FM, and EHVC. Notably, it brings an average of 2.84% and 4.05% bitrate savings in terms of PSNR and MS-SSIM, respectively, to DCVC-FM with negligible coding time increase. Our code is available at https://github.com/SunnyMass/SOFR.