Deep Multi-class Adversarial Specularity Removal
This addresses image processing challenges for computer vision applications, but it is incremental as it builds on existing GAN frameworks with modifications.
The paper tackles the problem of removing specular highlights from single images by generating diffuse components, achieving improved consistency over state-of-the-art methods.
We propose a novel learning approach, in the form of a fully-convolutional neural network (CNN), which automatically and consistently removes specular highlights from a single image by generating its diffuse component. To train the generative network, we define an adversarial loss on a discriminative network as in the GAN framework and combined it with a content loss. In contrast to existing GAN approaches, we implemented the discriminator to be a multi-class classifier instead of a binary one, to find more constraining features. This helps the network pinpoint the diffuse manifold by providing two more gradient terms. We also rendered a synthetic dataset designed to help the network generalize well. We show that our model performs well across various synthetic and real images and outperforms the state-of-the-art in consistency.