Recovering Continuous Scene Dynamics from A Single Blurry Image with Events
This addresses the challenge of temporal ambiguity in motion-blurred images for computer vision applications, representing an incremental improvement through hybrid modality fusion.
The paper tackles the problem of recovering continuous scene dynamics from a single motion-blurred image with event data, achieving state-of-the-art performance in sharp image restoration across arbitrary timestamps. Experiments show it outperforms existing methods by a large margin in PSNR and SSIM measurements.
This paper aims at demystifying a single motion-blurred image with events and revealing temporally continuous scene dynamics encrypted behind motion blurs. To achieve this end, an Implicit Video Function (IVF) is learned to represent a single motion blurred image with concurrent events, enabling the latent sharp image restoration of arbitrary timestamps in the range of imaging exposures. Specifically, a dual attention transformer is proposed to efficiently leverage merits from both modalities, i.e., the high temporal resolution of event features and the smoothness of image features, alleviating temporal ambiguities while suppressing the event noise. The proposed network is trained only with the supervision of ground-truth images of limited referenced timestamps. Motion- and texture-guided supervisions are employed simultaneously to enhance restorations of the non-referenced timestamps and improve the overall sharpness. Experiments on synthetic, semi-synthetic, and real-world datasets demonstrate that our proposed method outperforms state-of-the-art methods by a large margin in terms of both objective PSNR and SSIM measurements and subjective evaluations.