Visual Verity in AI-Generated Imagery: Computational Metrics and Human-Centric Analysis
This addresses the need for better evaluation methods for AI-generated imagery across sectors like entertainment and advertising, though it is incremental in refining existing metrics.
The paper tackled the problem of evaluating AI-generated images by developing and validating the Visual Verity questionnaire to measure photorealism, image quality, and text-image alignment, and found that camera-generated images outperformed AI models in photorealism and text-image alignment, while AI models led in image quality, with MS-SSIM and CLIP identified as computational metrics most aligned with human judgments.
The rapid advancements in AI technologies have revolutionized the production of graphical content across various sectors, including entertainment, advertising, and e-commerce. These developments have spurred the need for robust evaluation methods to assess the quality and realism of AI-generated images. To address this, we conducted three studies. First, we introduced and validated a questionnaire called Visual Verity, which measures photorealism, image quality, and text-image alignment. Second, we applied this questionnaire to assess images from AI models (DALL-E2, DALL-E3, GLIDE, Stable Diffusion) and camera-generated images, revealing that camera-generated images excelled in photorealism and text-image alignment, while AI models led in image quality. We also analyzed statistical properties, finding that camera-generated images scored lower in hue, saturation, and brightness. Third, we evaluated computational metrics' alignment with human judgments, identifying MS-SSIM and CLIP as the most consistent with human assessments. Additionally, we proposed the Neural Feature Similarity Score (NFSS) for assessing image quality. Our findings highlight the need for refining computational metrics to better capture human visual perception, thereby enhancing AI-generated content evaluation.