PyNET-QxQ: An Efficient PyNET Variant for QxQ Bayer Pattern Demosaicing in CMOS Image Sensors
This work addresses the need for efficient demosaicing models for mobile cameras using QxQ Bayer patterns, though it is incremental as it adapts an existing method to a specific domain.
The authors tackled the problem of high computational demands in deep learning-based image signal processors for mobile cameras with non-Bayer color filter arrays, specifically QxQ Bayer patterns, by developing PyNET-QxQ, a lightweight variant with less than 2.5% of the parameters of the original PyNET while preserving performance and outperforming existing conventional algorithms in texture and edge reconstruction.
Deep learning-based image signal processor (ISP) models for mobile cameras can generate high-quality images that rival those of professional DSLR cameras. However, their computational demands often make them unsuitable for mobile settings. Additionally, modern mobile cameras employ non-Bayer color filter arrays (CFA) such as Quad Bayer, Nona Bayer, and QxQ Bayer to enhance image quality, yet most existing deep learning-based ISP (or demosaicing) models focus primarily on standard Bayer CFAs. In this study, we present PyNET-QxQ, a lightweight demosaicing model specifically designed for QxQ Bayer CFA patterns, which is derived from the original PyNET. We also propose a knowledge distillation method called progressive distillation to train the reduced network more effectively. Consequently, PyNET-QxQ contains less than 2.5% of the parameters of the original PyNET while preserving its performance. Experiments using QxQ images captured by a proto type QxQ camera sensor show that PyNET-QxQ outperforms existing conventional algorithms in terms of texture and edge reconstruction, despite its significantly reduced parameter count.