GenExam: A Multidisciplinary Text-to-Image Exam
This addresses the need for a more challenging benchmark to assess AI models' integrated understanding, reasoning, and generation abilities in image creation, though it is incremental as it builds on existing evaluation frameworks.
The authors tackled the problem of evaluating text-to-image models on rigorous drawing exams by introducing GenExam, a multidisciplinary benchmark with 1,000 samples across 10 subjects, and found that state-of-the-art models achieve less than 15% strict scores, with most scoring near 0%.
Exams are a fundamental test of expert-level intelligence and require integrated understanding, reasoning, and generation. Existing exam-style benchmarks mainly focus on understanding and reasoning tasks, and current generation benchmarks emphasize the illustration of world knowledge and visual concepts, neglecting the evaluation of rigorous drawing exams. We introduce GenExam, the first benchmark for multidisciplinary text-to-image exams, featuring 1,000 samples across 10 subjects with exam-style prompts organized under a four-level taxonomy. Each problem is equipped with ground-truth images and fine-grained scoring points to enable a precise evaluation of semantic correctness and visual plausibility. Experiments show that even state-of-the-art models such as GPT-Image-1 and Gemini-2.5-Flash-Image achieve less than 15% strict scores, and most models yield almost 0%, suggesting the great challenge of our benchmark. By framing image generation as an exam, GenExam offers a rigorous assessment of models' ability to integrate understanding, reasoning, and generation, providing insights on the path to general AGI. Our benchmark and evaluation code are released at https://github.com/OpenGVLab/GenExam.