LiveEvalBench: Toward Open-World Evaluation for Web Generation
For researchers and developers of web generation models, this provides a more realistic evaluation framework that addresses the dynamic and diverse nature of frontend artifacts, but it is an incremental step in evaluation methodology.
The paper introduces LiveEvalBench, an automated framework for evaluating web generation that treats it as an interactive, adaptive process rather than a static benchmark. The framework aligns closely with human expert judgment and provides fine-grained insights into frontier models' capabilities, though no specific numerical results are provided.
Large language models are increasingly capable of synthesizing executable frontend projects, yet existing benchmarks still treat web generation as a static evaluation problem. We argue that frontend artifacts demand a different paradigm: they are interactive rather than static, admit diverse yet equally valid implementations, and evolve faster than rigid pipelines can accommodate. To address these gaps, we present LiveEvalBench, an automated framework that reformulates web-generation evaluation as an agentic, adaptive, and extensible process. LiveEvalBench instantiates evaluation as a collaborative review workflow, in which a Build Engineer, a Code Engineer, and a UI Tester collectively gather evidence across the full lifecycle of a frontend project, from deployment and code inspection to browser-based interaction. To handle implementation diversity, an adaptive protocol couples shared rubrics for cross-model comparability with implementation-grounded criteria tailored to each artifact. The framework further supports incremental integration of new evaluator roles and assessment dimensions without pipeline redesign. Experiments across diverse real-world web-generation scenarios show that LiveEvalBench aligns closely with human expert judgment and provides fine-grained insights into frontier models' web generation capabilities. Code is available at https://github.com/wyysteelhead/LiveEvalBench