A HVS-inspired Attention to Improve Loss Metrics for CNN-based Perception-Oriented Super-Resolution
This work addresses the challenge of enhancing visual fidelity in super-resolution for applications like image processing, though it is incremental as it builds on existing perceptual loss methods.
The paper tackled the problem of improving perceptual quality in CNN-based super-resolution by proposing a spatial attention mechanism inspired by the Human Visual System, which led to more natural images as demonstrated by improved performance of perceptual and contextual losses.
Deep Convolutional Neural Network (CNN) features have been demonstrated to be effective perceptual quality features. The perceptual loss, based on feature maps of pre-trained CNN's has proven to be remarkably effective for CNN based perceptual image restoration problems. In this work, taking inspiration from the the Human Visual System (HVS) and visual perception, we propose a spatial attention mechanism based on the dependency human contrast sensitivity on spatial frequency. We identify regions in input images, based on the underlying spatial frequency, which are not generally well reconstructed during Super-Resolution but are most important in terms of visual sensitivity. Based on this prior, we design a spatial attention map that is applied to feature maps in the perceptual loss and its variants, helping them to identify regions that are of more perceptual importance. The results demonstrate the our technique improves the ability of the perceptual loss and contextual loss to deliver more natural images in CNN based super-resolution.