Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes
This addresses a critical issue for CMOS camera users by enabling better video quality in dynamic environments, though it is incremental as it builds on prior work with a new dataset and model.
The paper tackles the joint problem of rolling shutter correction and deblurring in dynamic scenes, where existing methods fail, by introducing the first dataset (BS-RSCD) and a learning-based model (JCD) that achieves state-of-the-art performance on realistic and synthetic datasets.
Joint rolling shutter correction and deblurring (RSCD) techniques are critical for the prevalent CMOS cameras. However, current approaches are still based on conventional energy optimization and are developed for static scenes. To enable learning-based approaches to address real-world RSCD problem, we contribute the first dataset, BS-RSCD, which includes both ego-motion and object-motion in dynamic scenes. Real distorted and blurry videos with corresponding ground truth are recorded simultaneously via a beam-splitter-based acquisition system. Since direct application of existing individual rolling shutter correction (RSC) or global shutter deblurring (GSD) methods on RSCD leads to undesirable results due to inherent flaws in the network architecture, we further present the first learning-based model (JCD) for RSCD. The key idea is that we adopt bi-directional warping streams for displacement compensation, while also preserving the non-warped deblurring stream for details restoration. The experimental results demonstrate that JCD achieves state-of-the-art performance on the realistic RSCD dataset (BS-RSCD) and the synthetic RSC dataset (Fastec-RS). The dataset and code are available at https://github.com/zzh-tech/RSCD.