Enhanced Correlation Matching based Video Frame Interpolation
This addresses video frame interpolation for high-resolution content like 4K videos, offering improved quality with fewer parameters, though it appears incremental as it builds on existing correlation and flow-based methods.
The paper tackles video frame interpolation for high-resolution 4K videos with large motion and occlusion, proposing a DNN framework that uses enhanced correlation matching and recurrent pyramid architecture. The result shows it outperforms previous methods on 4K and benchmark datasets in objective and subjective quality with the smallest model parameters.
We propose a novel DNN based framework called the Enhanced Correlation Matching based Video Frame Interpolation Network to support high resolution like 4K, which has a large scale of motion and occlusion. Considering the extensibility of the network model according to resolution, the proposed scheme employs the recurrent pyramid architecture that shares the parameters among each pyramid layer for optical flow estimation. In the proposed flow estimation, the optical flows are recursively refined by tracing the location with maximum correlation. The forward warping based correlation matching enables to improve the accuracy of flow update by excluding incorrectly warped features around the occlusion area. Based on the final bi-directional flows, the intermediate frame at arbitrary temporal position is synthesized using the warping and blending network and it is further improved by refinement network. Experiment results demonstrate that the proposed scheme outperforms the previous works at 4K video data and low-resolution benchmark datasets as well in terms of objective and subjective quality with the smallest number of model parameters.