Single Image Rolling Shutter Removal with Diffusion Models
This work addresses the important but difficult task of correcting rolling shutter artifacts from single images, which is incremental as it applies diffusion models to a specific domain problem.
The paper tackles the problem of single-frame rolling shutter correction, which is challenging due to row-wise exposure artifacts in CMOS sensors, and introduces RS-Diffusion, a diffusion-based method that outperforms previous single-frame approaches and includes a new dataset, RS-Real, with captured RS and corrected GS ground-truth pairs.
We present RS-Diffusion, the first Diffusion Models-based method for single-frame Rolling Shutter (RS) correction. RS artifacts compromise visual quality of frames due to the row-wise exposure of CMOS sensors. Most previous methods have focused on multi-frame approaches, using temporal information from consecutive frames for the motion rectification. However, few approaches address the more challenging but important single frame RS correction. In this work, we present an ``image-to-motion" framework via diffusion techniques, with a designed patch-attention module. In addition, we present the RS-Real dataset, comprised of captured RS frames alongside their corresponding Global Shutter (GS) ground-truth pairs. The GS frames are corrected from the RS ones, guided by the corresponding Inertial Measurement Unit (IMU) gyroscope data acquired during capture. Experiments show that RS-Diffusion surpasses previous single-frame RS methods, demonstrates the potential of diffusion-based approaches, and provides a valuable dataset for further research.