DeepHist: Differentiable Joint and Color Histogram Layers for Image-to-Image Translation
This work addresses image-to-image translation for computer vision applications, offering a method to precisely control color distributions, though it is incremental as it builds on existing techniques like Pix2Pix and CycleGANs.
The paper tackles the problem of image-to-image translation by controlling the color distribution of output images, introducing a novel framework that uses differentiable histogram layers to match reference color distributions while preserving structural content, achieving promising results in tasks like color transfer and image colorization.
We present the DeepHist - a novel Deep Learning framework for augmenting a network by histogram layers and demonstrate its strength by addressing image-to-image translation problems. Specifically, given an input image and a reference color distribution we aim to generate an output image with the structural appearance (content) of the input (source) yet with the colors of the reference. The key idea is a new technique for a differentiable construction of joint and color histograms of the output images. We further define a color distribution loss based on the Earth Mover's Distance between the output's and the reference's color histograms and a Mutual Information loss based on the joint histograms of the source and the output images. Promising results are shown for the tasks of color transfer, image colorization and edges $\rightarrow$ photo, where the color distribution of the output image is controlled. Comparison to Pix2Pix and CyclyGANs are shown.