Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network
This addresses a fundamental limitation in computer vision tasks requiring physically accurate radiance values for images taken in automatic camera mode.
The paper tackles the problem of modeling scene-dependent camera image processing (radiometric calibration) for automatic camera mode images, where previous methods only worked for manual mode. The proposed deep learning framework accurately models different cameras' imaging pipelines in both RAW-to-sRGB and sRGB-to-RAW directions and demonstrates improved image deblurring performance.
We present a novel deep learning framework that models the scene dependent image processing inside cameras. Often called as the radiometric calibration, the process of recovering RAW images from processed images (JPEG format in the sRGB color space) is essential for many computer vision tasks that rely on physically accurate radiance values. All previous works rely on the deterministic imaging model where the color transformation stays the same regardless of the scene and thus they can only be applied for images taken under the manual mode. In this paper, we propose a data-driven approach to learn the scene dependent and locally varying image processing inside cameras under the automode. Our method incorporates both the global and the local scene context into pixel-wise features via multi-scale pyramid of learnable histogram layers. The results show that we can model the imaging pipeline of different cameras that operate under the automode accurately in both directions (from RAW to sRGB, from sRGB to RAW) and we show how we can apply our method to improve the performance of image deblurring.