UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents
This addresses the need for UX researchers to pre-test study designs efficiently, though it is an incremental application of existing LLM agent technology to a new domain.
The authors tackled the problem of evaluating usability testing study designs before conducting human-subject studies by developing UXAgent, a system that uses LLM agents to simulate thousands of users interacting with websites, which was positively received in a heuristic evaluation with 16 UX researchers.
Usability testing is a fundamental research method that user experience (UX) researchers use to evaluate and iterate their new designs. But what about evaluating and iterating the usability testing study design itself? Recent advances in Large Language Model-simulated Agent (LLM Agent) research inspired us to design UXAgent to support UX researchers in evaluating and iterating their study design before they conduct the real human-subject study. Our system features a Persona Generator module, an LLM Agent module, and a Universal Browser Connector module to automatically generate thousands of simulated users and to interactively test the target website. The system also provides a Result Viewer Interface so that the UX researchers can easily review and analyze the generated qualitative (e.g., agents' post-study surveys) and quantitative data (e.g., agents' interaction logs), or even interview agents directly. Through a heuristic evaluation with 16 UX researchers, participants praised the innovation of our system but also expressed concerns about the future of LLM Agent usage in UX studies.