Contextual colorization and denoising for low-light ultra high resolution sequences
This work provides a method for improving the visual quality of low-light ultra high resolution image sequences, which is beneficial for applications where ground truth data is unavailable.
The paper addresses the challenge of enhancing low-light ultra high resolution image sequences, which typically suffer from noise, flicker, and motion blur. They propose an unpaired-learning method, based on a multiscale patch-based CycleGAN adaptation with adaptive temporal smoothing, to simultaneously colorize and denoise these sequences.
Low-light image sequences generally suffer from spatio-temporal incoherent noise, flicker and blurring of moving objects. These artefacts significantly reduce visual quality and, in most cases, post-processing is needed in order to generate acceptable quality. Most state-of-the-art enhancement methods based on machine learning require ground truth data but this is not usually available for naturally captured low light sequences. We tackle these problems with an unpaired-learning method that offers simultaneous colorization and denoising. Our approach is an adaptation of the CycleGAN structure. To overcome the excessive memory limitations associated with ultra high resolution content, we propose a multiscale patch-based framework, capturing both local and contextual features. Additionally, an adaptive temporal smoothing technique is employed to remove flickering artefacts. Experimental results show that our method outperforms existing approaches in terms of subjective quality and that it is robust to variations in brightness levels and noise.