CDFI: Compression-Driven Network Design for Frame Interpolation
This work addresses the deployment challenge for frame interpolation on mobile devices, representing an incremental improvement through compression and architectural enhancements.
The paper tackles the problem of deploying DNN-based frame interpolation on resource-limited devices by proposing CDFI, which uses model pruning and a multi-resolution warping module to reduce model size by 75% while achieving superior performance compared to the original AdaCoF and other state-of-the-art methods.
DNN-based frame interpolation--that generates the intermediate frames given two consecutive frames--typically relies on heavy model architectures with a huge number of features, preventing them from being deployed on systems with limited resources, e.g., mobile devices. We propose a compression-driven network design for frame interpolation (CDFI), that leverages model pruning through sparsity-inducing optimization to significantly reduce the model size while achieving superior performance. Concretely, we first compress the recently proposed AdaCoF model and show that a 10X compressed AdaCoF performs similarly as its original counterpart; then we further improve this compressed model by introducing a multi-resolution warping module, which boosts visual consistencies with multi-level details. As a consequence, we achieve a significant performance gain with only a quarter in size compared with the original AdaCoF. Moreover, our model performs favorably against other state-of-the-arts in a broad range of datasets. Finally, the proposed compression-driven framework is generic and can be easily transferred to other DNN-based frame interpolation algorithm. Our source code is available at https://github.com/tding1/CDFI.