Zoom-In-to-Check: Boosting Video Interpolation via Instance-level Discrimination
This work addresses video interpolation for applications like video editing or compression, offering an incremental improvement through a novel supervision method.
The paper tackles video frame interpolation by introducing instance-level supervision that learns from high-resolution objects, achieving state-of-the-art results across datasets with reduced computational resources.
We propose a light-weight video frame interpolation algorithm. Our key innovation is an instance-level supervision that allows information to be learned from the high-resolution version of similar objects. Our experiment shows that the proposed method can generate state-of-the-art results across different datasets, with fractional computation resources (time and memory) of competing methods. Given two image frames, a cascade network creates an intermediate frame with 1) a flow-warping module that computes coarse bi-directional optical flow and creates an interpolated image via flow-based warping, followed by 2) an image synthesis module to make fine-scale corrections. In the learning stage, object detection proposals are generated on the interpolated image.Lower resolution objects are zoomed into, and the learning algorithms using an adversarial loss trained on high-resolution objects to guide the system towards the instance-level refinement corrects details of object shape and boundaries.