Joint Demosaicking and Denoising Benefits from a Two-stage Training Strategy
This addresses image quality issues in color image production pipelines, offering a novel approach to a specific bottleneck in demosaicking and denoising.
The paper tackles the problem of oversmoothing and checkerboard effects in image demosaicking and denoising by inverting the traditional pipeline to demosaick first and then denoise, using a two-stage training strategy with CNNs, resulting in state-of-the-art improvements in quantitative and visual quality.
Image demosaicking and denoising are the first two key steps of the color image production pipeline. The classical processing sequence has for a long time consisted of applying denoising first, and then demosaicking. Applying the operations in this order leads to oversmoothing and checkerboard effects. Yet, it was difficult to change this order, because once the image is demosaicked, the statistical properties of the noise are dramatically changed and hard to handle by traditional denoising models. In this paper, we address this problem by a hybrid machine learning method. We invert the traditional color filter array (CFA) processing pipeline by first demosaicking and then denoising. Our demosaicking algorithm, trained on noiseless images, combines a traditional method and a residual convolutional neural network (CNN). This first stage retains all known information, which is the key point to obtain faithful final results. The noisy demosaicked image is then passed through a second CNN restoring a noiseless full-color image. This pipeline order completely avoids checkerboard effects and restores fine image detail. Although CNNs can be trained to solve jointly demosaicking-denoising end-to-end, we find that this two-stage training performs better and is less prone to failure. It is shown experimentally to improve on the state of the art, both quantitatively and in terms of visual quality.