Hierarchical Flow Diffusion for Efficient Frame Interpolation
This addresses the efficiency and accuracy gap in diffusion-based video frame interpolation for applications like video processing and enhancement.
The paper tackles the problem of video frame interpolation by proposing a hierarchical diffusion model that explicitly models bilateral optical flow, achieving state-of-the-art accuracy and being over 10 times faster than other diffusion-based methods.
Most recent diffusion-based methods still show a large gap compared to non-diffusion methods for video frame interpolation, in both accuracy and efficiency. Most of them formulate the problem as a denoising procedure in latent space directly, which is less effective caused by the large latent space. We propose to model bilateral optical flow explicitly by hierarchical diffusion models, which has much smaller search space in the denoising procedure. Based on the flow diffusion model, we then use a flow-guided images synthesizer to produce the final result. We train the flow diffusion model and the image synthesizer end to end. Our method achieves state of the art in accuracy, and 10+ times faster than other diffusion-based methods. The project page is at: https://hfd-interpolation.github.io.