DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning
This work addresses camera ISP optimization for vision tasks, offering a domain-specific incremental improvement by integrating DRL with existing ISP tools.
The paper tackles the problem of camera image signal processing (ISP) by proposing a multi-objective framework using deep reinforcement learning (DRL) to iteratively select and apply ISP tools from a toolbox of 51 options, improving image quality for tasks like RAW-to-RGB restoration, object detection, and depth estimation.
In this paper, we propose a multi-objective camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox that consist of network-based and conventional ISP tools. The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function. For this purpose, we implement total 51 ISP tools that include exposure correction, color-and-tone correction, white balance, sharpening, denoising, and the others. We also propose an efficient DRL network architecture that can extract the various aspects of an image and make a rigid mapping relationship between images and a large number of actions. Our proposed DRL-based ISP framework effectively improves the image quality according to each vision task such as RAW-to-RGB image restoration, 2D object detection, and monocular depth estimation.