HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models
This work is significant for researchers and practitioners working with HDR video reconstruction from dual-exposure sensors, offering a solution to the data scarcity problem for training denoising and deblurring models.
The paper addresses the challenge of reconstructing sharp and noise-free high-dynamic range (HDR) video from a dual-exposure sensor that provides noisy low-exposure data in odd columns and motion-blurred high-exposure data in even columns. The authors propose learning a CLEAN->DISTORTED function to generate training data, overcoming the scarcity of CLEAN HDR videos by learning from LDR video instead. Their method outperforms several strong baselines and can enhance existing methods when re-trained on their generated data.
We seek to reconstruct sharp and noise-free high-dynamic range (HDR) video from a dual-exposure sensor that records different low-dynamic range (LDR) information in different pixel columns: Odd columns provide low-exposure, sharp, but noisy information; even columns complement this with less noisy, high-exposure, but motion-blurred data. Previous LDR work learns to deblur and denoise (DISTORTED->CLEAN) supervised by pairs of CLEAN and DISTORTED images. Regrettably, capturing DISTORTED sensor readings is time-consuming; as well, there is a lack of CLEAN HDR videos. We suggest a method to overcome those two limitations. First, we learn a different function instead: CLEAN->DISTORTED, which generates samples containing correlated pixel noise, and row and column noise, as well as motion blur from a low number of CLEAN sensor readings. Second, as there is not enough CLEAN HDR video available, we devise a method to learn from LDR video in-stead. Our approach compares favorably to several strong baselines, and can boost existing methods when they are re-trained on our data. Combined with spatial and temporal super-resolution, it enables applications such as re-lighting with low noise or blur.