ChatGPT and U(X): A Rapid Review on Measuring the User Experience
This work addresses the lack of standardized UX evaluation methods for ChatGPT, which is used by millions, by synthesizing existing approaches to help researchers and practitioners improve user interactions, though it is incremental as it reviews and organizes prior studies.
The paper conducted a rapid review of 58 studies to analyze how user experience (UX) with ChatGPT is measured, identifying trends, gaps, and emerging consensus in assessments. It provides preliminary frameworks to guide future research and tool development for optimizing interactions with ChatGPT and similar LLM-based systems.
ChatGPT, powered by a large language model (LLM), has revolutionized everyday human-computer interaction (HCI) since its 2022 release. While now used by millions around the world, a coherent pathway for evaluating the user experience (UX) ChatGPT offers remains missing. In this rapid review (N = 58), I explored how ChatGPT UX has been approached quantitatively so far. I focused on the independent variables (IVs) manipulated, the dependent variables (DVs) measured, and the methods used for measurement. Findings reveal trends, gaps, and emerging consensus in UX assessments. This work offers a first step towards synthesizing existing approaches to measuring ChatGPT UX, urgent trajectories to advance standardization and breadth, and two preliminary frameworks aimed at guiding future research and tool development. I seek to elevate the field of ChatGPT UX by empowering researchers and practitioners in optimizing user interactions with ChatGPT and similar LLM-based systems.