Raw or Cooked? Object Detection on RAW Images
This work addresses the inefficiency of handcrafted ISP for downstream tasks like object detection, offering a domain-specific improvement for computer vision applications.
The paper tackles the problem of suboptimal image signal processing (ISP) for computer vision tasks by proposing a learnable ISP operation optimized for object detection, achieving superior performance compared to previous methods and traditional RGB images on the PASCALRAW dataset.
Images fed to a deep neural network have in general undergone several handcrafted image signal processing (ISP) operations, all of which have been optimized to produce visually pleasing images. In this work, we investigate the hypothesis that the intermediate representation of visually pleasing images is sub-optimal for downstream computer vision tasks compared to the RAW image representation. We suggest that the operations of the ISP instead should be optimized towards the end task, by learning the parameters of the operations jointly during training. We extend previous works on this topic and propose a new learnable operation that enables an object detector to achieve superior performance when compared to both previous works and traditional RGB images. In experiments on the open PASCALRAW dataset, we empirically confirm our hypothesis.