User Simulation in the Era of Generative AI: User Modeling, Synthetic Data Generation, and System Evaluation
It addresses the need for controlled and reproducible methods to model user behavior and evaluate AI systems, which is crucial for researchers and developers in fields like AI and human-computer interaction, but it is incremental as it primarily reviews existing concepts.
This paper tackles the problem of simulating human users for AI systems by providing an overview of user simulation, which involves creating intelligent agents to mimic user actions for modeling behavior, generating synthetic data, and evaluating interactive AI systems, with a focus on its applications and future directions in the era of Generative AI.
User simulation is an emerging interdisciplinary topic with multiple critical applications in the era of Generative AI. It involves creating an intelligent agent that mimics the actions of a human user interacting with an AI system, enabling researchers to model and analyze user behaviour, generate synthetic data for training, and evaluate interactive AI systems in a controlled and reproducible manner. User simulation has profound implications for diverse fields and plays a vital role in the pursuit of Artificial General Intelligence. This paper provides an overview of user simulation, highlighting its key applications, connections to various disciplines, and outlining future research directions to advance this increasingly important technology.