Perceptually Motivated Method for Image Inpainting Comparison
This addresses the problem of inconsistent evaluation for researchers in computer vision, though it is incremental as it builds on existing metrics.
The paper tackled the lack of a standard evaluation method for image inpainting algorithms by conducting a subjective comparison of nine state-of-the-art methods and proposing objective metrics that correlate well with human perception.
The field of automatic image inpainting has progressed rapidly in recent years, but no one has yet proposed a standard method of evaluating algorithms. This absence is due to the problem's challenging nature: image-inpainting algorithms strive for realism in the resulting images, but realism is a subjective concept intrinsic to human perception. Existing objective image-quality metrics provide a poor approximation of what humans consider more or less realistic. To improve the situation and to better organize both prior and future research in this field, we conducted a subjective comparison of nine state-of-the-art inpainting algorithms and propose objective quality metrics that exhibit high correlation with the results of our comparison.