RAN4IQA: Restorative Adversarial Nets for No-Reference Image Quality Assessment
This addresses image quality assessment for applications like photography and video processing, but it is incremental as it builds on existing GAN and perceptual modeling approaches.
The paper tackled no-reference image quality assessment by proposing a GAN-based model that mimics the human visual system to restore distorted images and measure perceptual quality, achieving state-of-the-art results on datasets like Waterloo Exploration, LIVE, and TID2013.
Inspired by the free-energy brain theory, which implies that human visual system (HVS) tends to reduce uncertainty and restore perceptual details upon seeing a distorted image, we propose restorative adversarial net (RAN), a GAN-based model for no-reference image quality assessment (NR-IQA). RAN, which mimics the process of HVS, consists of three components: a restorator, a discriminator and an evaluator. The restorator restores and reconstructs input distorted image patches, while the discriminator distinguishes the reconstructed patches from the pristine distortion-free patches. After restoration, we observe that the perceptual distance between the restored and the distorted patches is monotonic with respect to the distortion level. We further define Gain of Restoration (GoR) based on this phenomenon. The evaluator predicts perceptual score by extracting feature representations from the distorted and restored patches to measure GoR. Eventually, the quality score of an input image is estimated by weighted sum of the patch scores. Experimental results on Waterloo Exploration, LIVE and TID2013 show the effectiveness and generalization ability of RAN compared to the state-of-the-art NR-IQA models.