Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation
This work provides an incremental improvement for video frame interpolation, specifically benefiting applications that require handling large temporal gaps and complex motion, such as slow-motion video generation or high-speed scene analysis.
This paper addresses the challenge of video frame interpolation for large temporal gaps and complex motion by integrating event camera data into pre-trained latent diffusion models. By using Image Warped Events (IWEs) and bidirectional sparse optical flow as guidance, the method reduces interpolation artifacts and improves reconstruction fidelity and temporal coherence, outperforming existing state-of-the-art approaches on real and synthetic benchmarks.
Latent diffusion models have recently advanced video frame interpolation by synthesizing intermediate frames between input images. However, handling large temporal gaps and complex motion remains challenging, often resulting in motion blur, structural distortions, and temporal inconsistencies. Event cameras provide high-temporal-resolution motion cues that are well suited for bridging these gaps and improving interpolation quality. To exploit this advantage without training an event-assisted model from scratch, we propose an adapter-based framework that incorporates event-derived cues into a pre-trained image-to-video diffusion model with minimal architectural changes. Specifically, our method leverages Image Warped Events (IWEs) and bidirectional sparse optical flow to provide spatially and temporally aligned guidance during generation. By injecting these event-guided structural and motion cues into the diffusion process, our approach reduces interpolation artifacts and improves both reconstruction fidelity and temporal coherence. Experimental results on real and synthetic benchmarks show that our method consistently outperforms existing state-of-the-art approaches. The project page is at https://joseph-lin-tech.github.io/BridgeEventDiT-VFI/.