Ambiguity of Objective Image Quality Metrics: A New Methodology for Performance Evaluation
This addresses a gap in image quality assessment for researchers and practitioners by providing a new evaluation criterion, though it is incremental as it builds on existing metrics and databases.
The paper tackles the problem that objective image quality metrics can produce different scores for images that are perceptually indistinguishable to humans, proposing a method to define an ambiguity interval for these metrics. The results demonstrate that this interval can serve as an additional performance measure, tested on 33 state-of-the-art metrics across three databases.
Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment. In this paper, we address the issue of ambiguity of objective image quality assessment. We propose an approach to obtain an ambiguity interval of an objective metric, within which the quality score difference is not perceptually significant. In particular, we use the visual difference predictor, which can consider viewing conditions that are important for visual quality perception. In order to demonstrate the usefulness of the proposed approach, we conduct experiments with 33 state-of-the-art image quality metrics in the viewpoint of their accuracy and ambiguity for three image quality databases. The results show that the ambiguity intervals can be applied as an additional figure of merit when conventional performance measurement does not determine superiority between the metrics. The effect of the viewing distance on the ambiguity interval is also shown.