UI-Bench: A Benchmark for Evaluating Design Capabilities of AI Text-to-App Tools
This provides a benchmark for developers and researchers to evaluate and advance AI text-to-app tools, though it is incremental as it focuses on creating a standard rather than a new method.
The authors tackled the lack of rigorous evaluation for AI text-to-app tools by introducing UI-Bench, a large-scale benchmark that uses expert pairwise comparisons to rank 10 tools based on 300 generated sites and over 4,000 judgments, establishing a reproducible standard for AI-driven web design.
AI text-to-app tools promise high quality applications and websites in minutes, yet no public benchmark rigorously verifies those claims. We introduce UI-Bench, the first large-scale benchmark that evaluates visual excellence across competing AI text-to-app tools through expert pairwise comparison. Spanning 10 tools, 30 prompts, 300 generated sites, and 4,000+ expert judgments, UI-Bench ranks systems with a TrueSkill-derived model that yields calibrated confidence intervals. UI-Bench establishes a reproducible standard for advancing AI-driven web design. We release (i) the complete prompt set, (ii) an open-source evaluation framework, and (iii) a public leaderboard. The generated sites rated by participants will be released soon. View the UI-Bench leaderboard at https://uibench.ai/leaderboard.