Inheriting Bayer's Legacy-Joint Remosaicing and Denoising for Quad Bayer Image Sensor
This work addresses image quality issues in compact cameras using Quad sensors, offering a software solution to enhance low-light performance without hardware changes, though it is incremental as it builds on existing sensor technology and deep learning methods.
The paper tackles the problem of low-light imaging with Quad Bayer sensors, which suffer from reduced resolution and artifacts after pixel binning, by proposing a dual-head joint remosaicing and denoising network (DJRD) that converts noisy Quad Bayer to noise-free Bayer without resolution loss, outperforming competing models by approximately 3dB.
Pixel binning based Quad sensors have emerged as a promising solution to overcome the hardware limitations of compact cameras in low-light imaging. However, binning results in lower spatial resolution and non-Bayer CFA artifacts. To address these challenges, we propose a dual-head joint remosaicing and denoising network (DJRD), which enables the conversion of noisy Quad Bayer and standard noise-free Bayer pattern without any resolution loss. DJRD includes a newly designed Quad Bayer remosaicing (QB-Re) block, integrated denoising modules based on Swin-transformer and multi-scale wavelet transform. The QB-Re block constructs the convolution kernel based on the CFA pattern to achieve a periodic color distribution in the perceptual field, which is used to extract exact spectral information and reduce color misalignment. The integrated Swin-Transformer and multi-scale wavelet transform capture non-local dependencies, frequency and location information to effectively reduce practical noise. By identifying challenging patches utilizing Moire and zipper detection metrics, we enable our model to concentrate on difficult patches during the post-training phase, which enhances the model's performance in hard cases. Our proposed model outperforms competing models by approximately 3dB, without additional complexity in hardware or software.