FANeRV: Frequency Separation and Augmentation based Neural Representation for Video
This work addresses limitations in neural video representations for applications such as video processing, but it is incremental as it builds on existing NeRV methods with targeted enhancements.
The paper tackles the problem of vague reconstructions in neural video representations by proposing FANeRV, which uses frequency separation and augmentation to improve fine spatial details, resulting in significant performance gains in tasks like video compression, inpainting, and interpolation.
Neural representations for video (NeRV) have gained considerable attention for their strong performance across various video tasks. However, existing NeRV methods often struggle to capture fine spatial details, resulting in vague reconstructions. In this paper, we present a Frequency Separation and Augmentation based Neural Representation for video (FANeRV), which addresses these limitations with its core Wavelet Frequency Upgrade Block. This block explicitly separates input frames into high and low-frequency components using discrete wavelet transform, followed by targeted enhancement using specialized modules. Finally, a specially designed gated network effectively fuses these frequency components for optimal reconstruction. Additionally, convolutional residual enhancement blocks are integrated into the later stages of the network to balance parameter distribution and improve the restoration of high-frequency details. Experimental results demonstrate that FANeRV significantly improves reconstruction performance and excels in multiple tasks, including video compression, inpainting, and interpolation, outperforming existing NeRV methods.