What Else Can Fool Deep Learning? Addressing Color Constancy Errors on Deep Neural Network Performance
This addresses a global image manipulation issue for deep learning practitioners, but it is incremental as it builds on existing augmentation and pre-processing techniques.
The paper tackles the problem of color constancy errors, specifically incorrect white balance, which negatively impacts deep neural network performance in image segmentation and classification, and demonstrates notable improvements on datasets like CIFAR-10, CIFAR-100, and ADE20K by proposing a novel augmentation method and using pre-processing strategies.
There is active research targeting local image manipulations that can fool deep neural networks (DNNs) into producing incorrect results. This paper examines a type of global image manipulation that can produce similar adverse effects. Specifically, we explore how strong color casts caused by incorrectly applied computational color constancy - referred to as white balance (WB) in photography - negatively impact the performance of DNNs targeting image segmentation and classification. In addition, we discuss how existing image augmentation methods used to improve the robustness of DNNs are not well suited for modeling WB errors. To address this problem, a novel augmentation method is proposed that can emulate accurate color constancy degradation. We also explore pre-processing training and testing images with a recent WB correction algorithm to reduce the effects of incorrectly white-balanced images. We examine both augmentation and pre-processing strategies on different datasets and demonstrate notable improvements on the CIFAR-10, CIFAR-100, and ADE20K datasets.