Video Frame Interpolation Based on Deformable Kernel Region
This work improves video frame interpolation for computer vision applications, but it is incremental as it builds on existing deep learning methods by refining kernel region handling.
The paper tackles the problem of video frame interpolation by addressing grid restrictions in kernel regions that limit adaptation to irregular object shapes and motion uncertainty, proposing a deformable convolution approach that achieves superior performance on four datasets.
Video frame interpolation task has recently become more and more prevalent in the computer vision field. At present, a number of researches based on deep learning have achieved great success. Most of them are either based on optical flow information, or interpolation kernel, or a combination of these two methods. However, these methods have ignored that there are grid restrictions on the position of kernel region during synthesizing each target pixel. These limitations result in that they cannot well adapt to the irregularity of object shape and uncertainty of motion, which may lead to irrelevant reference pixels used for interpolation. In order to solve this problem, we revisit the deformable convolution for video interpolation, which can break the fixed grid restrictions on the kernel region, making the distribution of reference points more suitable for the shape of the object, and thus warp a more accurate interpolation frame. Experiments are conducted on four datasets to demonstrate the superior performance of the proposed model in comparison to the state-of-the-art alternatives.