Spectrally Consistent UNet for High Fidelity Image Transformations
This addresses image quality issues in computer vision applications like dynamic range expansion and colorization, but is incremental as it modifies an existing architecture.
The paper tackled artefacts in UNet architectures caused by upsampling layers by analyzing structural biases in the Fourier domain and proposing a Guided UNet (GUNet) with a new upsampling module using the Guided Image Filter, resulting in higher fidelity outputs for inverse tone mapping and colorization tasks.
Convolutional Neural Networks (CNNs) are the current de-facto models used for many imaging tasks due to their high learning capacity as well as their architectural qualities. The ubiquitous UNet architecture provides an efficient and multi-scale solution that combines local and global information. Despite the success of UNet architectures, the use of upsampling layers can cause artefacts. In this work, a method for assessing the structural biases of UNets and the effects these have on the outputs is presented, characterising their impact in the Fourier domain. A new upsampling module is proposed, based on a novel use of the Guided Image Filter, that provides spectrally consistent outputs when used in a UNet architecture, forming the Guided UNet (GUNet). The GUNet architecture is applied and evaluated for example applications of inverse tone mapping/dynamic range expansion and colourisation from grey-scale images and is shown to provide higher fidelity outputs.