AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation
This addresses the problem of generating intermediate video frames for applications like video editing or slow-motion, with incremental improvements in accuracy and efficiency over existing methods.
The paper tackles video frame interpolation by proposing AMT, a network architecture that uses bidirectional correlation volumes and multi-field transforms to handle large motions and occlusions, achieving state-of-the-art performance on various benchmarks with high efficiency.
We present All-Pairs Multi-Field Transforms (AMT), a new network architecture for video frame interpolation. It is based on two essential designs. First, we build bidirectional correlation volumes for all pairs of pixels, and use the predicted bilateral flows to retrieve correlations for updating both flows and the interpolated content feature. Second, we derive multiple groups of fine-grained flow fields from one pair of updated coarse flows for performing backward warping on the input frames separately. Combining these two designs enables us to generate promising task-oriented flows and reduce the difficulties in modeling large motions and handling occluded areas during frame interpolation. These qualities promote our model to achieve state-of-the-art performance on various benchmarks with high efficiency. Moreover, our convolution-based model competes favorably compared to Transformer-based models in terms of accuracy and efficiency. Our code is available at https://github.com/MCG-NKU/AMT.