UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design
This addresses the problem of inflexible and resource-intensive usability testing for UX researchers, though it is an incremental application of existing LLM agent technology to a new domain.
The paper tackles the challenge of usability testing for web design by introducing UXAgent, an LLM agent-based framework that automatically generates simulated users to evaluate websites, providing qualitative, quantitative, and video results, with positive feedback from a heuristic evaluation involving five UX researchers.
Usability testing is a fundamental yet challenging (e.g., inflexible to iterate the study design flaws and hard to recruit study participants) research method for user experience (UX) researchers to evaluate a web design. Recent advances in Large Language Model-simulated Agent (LLM-Agent) research inspired us to design UXAgent to support UX researchers in evaluating and reiterating their usability testing study design before they conduct the real human subject study. Our system features an LLM-Agent module and a universal browser connector module so that UX researchers can automatically generate thousands of simulated users to test the target website. The results are shown in qualitative (e.g., interviewing how an agent thinks ), quantitative (e.g., # of actions), and video recording formats for UX researchers to analyze. Through a heuristic user evaluation with five UX researchers, participants praised the innovation of our system but also expressed concerns about the future of LLM Agent-assisted UX study.