UXBench: Measuring the Actionability of LLM-Generated UX Critiques
For UX researchers and practitioners, this benchmark provides the first controlled measure of whether LLM-generated critiques are actionable for downstream interface repair.
LLMs are increasingly used as UX judges, but no benchmark measures the actionability of their critiques. UXBench evaluates eight frontier models across ten web fixture families, finding that models differ in report actionability, exhibit distinct repair signatures, and trade leadership across surface categories.
Large language models (LLMs) are increasingly deployed as UX judges that inspect interfaces, diagnose usability problems, and propose repairs. Yet no controlled benchmark measures whether the resulting critiques are reliable and actionable across heterogeneous product surfaces. We introduce UXBench, a benchmark for evaluating LLMs as interaction-grounded UX judges. UXBench comprises local-first runnable web fixtures spanning ten product-surface families, paired with coverage-gated browser exploration that forces models to collect interaction evidence before reporting. Each judge model produces a structured UX report over seven rubric dimensions; report quality is measured by whether a fixed downstream repair agent can improve the interface based on the critique. We evaluate eight frontier models under both an automated repair-lift protocol and a blind human validation study. Results show that UX judging is neither saturated nor one dimensional: models differ meaningfully in report actionability, exhibit distinct rubric-level repair signatures, vary in fixture-level reliability, and trade leadership across surface categories