T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation
This work addresses the need for standardized evaluation of reasoning in text-to-image generation for researchers and developers, though it is incremental as it builds on existing benchmarking efforts.
The authors tackled the problem of evaluating reasoning capabilities in text-to-image models by proposing T2I-ReasonBench, a benchmark with four dimensions and a two-stage evaluation protocol, resulting in comprehensive performance analysis of various models.
We propose T2I-ReasonBench, a benchmark evaluating reasoning capabilities of text-to-image (T2I) models. It consists of four dimensions: Idiom Interpretation, Textual Image Design, Entity-Reasoning and Scientific-Reasoning. We propose a two-stage evaluation protocol to assess the reasoning accuracy and image quality. We benchmark various T2I generation models, and provide comprehensive analysis on their performances.