Video Interpolation by Event-driven Anisotropic Adjustment of Optical Flow
This work addresses the problem of generating intermediate frames in videos for applications like slow-motion playback, though it appears incremental as it builds on existing optical flow methods with event data.
The paper tackles video frame interpolation by proposing an event-driven method to adjust optical flows anisotropically, addressing complex real-world motion. It reports outperforming previous methods and advancing supervised event-based video interpolation.
Video frame interpolation is a challenging task due to the ever-changing real-world scene. Previous methods often calculate the bi-directional optical flows and then predict the intermediate optical flows under the linear motion assumptions, leading to isotropic intermediate flow generation. Follow-up research obtained anisotropic adjustment through estimated higher-order motion information with extra frames. Based on the motion assumptions, their methods are hard to model the complicated motion in real scenes. In this paper, we propose an end-to-end training method A^2OF for video frame interpolation with event-driven Anisotropic Adjustment of Optical Flows. Specifically, we use events to generate optical flow distribution masks for the intermediate optical flow, which can model the complicated motion between two frames. Our proposed method outperforms the previous methods in video frame interpolation, taking supervised event-based video interpolation to a higher stage.