VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation
This work addresses the need for explainable metrics in conditional image synthesis evaluation, offering a potential replacement for human judges, though it shows limitations with open-source models and editing tasks.
The paper tackled the problem of limited explainability in evaluating conditional image generation models by introducing VIEScore, a visual instruction-guided metric that uses multimodal large language models without training, achieving a Spearman correlation of 0.4 with human evaluations compared to a human-to-human correlation of 0.45.
In the rapidly advancing field of conditional image generation research, challenges such as limited explainability lie in effectively evaluating the performance and capabilities of various models. This paper introduces VIEScore, a Visual Instruction-guided Explainable metric for evaluating any conditional image generation tasks. VIEScore leverages general knowledge from Multimodal Large Language Models (MLLMs) as the backbone and does not require training or fine-tuning. We evaluate VIEScore on seven prominent tasks in conditional image tasks and found: (1) VIEScore (GPT4-o) achieves a high Spearman correlation of 0.4 with human evaluations, while the human-to-human correlation is 0.45. (2) VIEScore (with open-source MLLM) is significantly weaker than GPT-4o and GPT-4v in evaluating synthetic images. (3) VIEScore achieves a correlation on par with human ratings in the generation tasks but struggles in editing tasks. With these results, we believe VIEScore shows its great potential to replace human judges in evaluating image synthesis tasks.