Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre
This work addresses the challenge of evaluating conversational agents in real-world, multi-party settings for co-creative arts, though it is incremental as it applies existing methods to a new domain.
The study deployed Large Language Models (LLMs) in a month-long live theatre show at the Edinburgh Festival Fringe to investigate co-creation between human improvisers and conversational agents, finding that audience feedback showed evolving interest in AI-driven entertainment and performers expressed enthusiasm with varied satisfaction.
Social robotics researchers are increasingly interested in multi-party trained conversational agents. With a growing demand for real-world evaluations, our study presents Large Language Models (LLMs) deployed in a month-long live show at the Edinburgh Festival Fringe. This case study investigates human improvisers co-creating with conversational agents in a professional theatre setting. We explore the technical capabilities and constraints of on-the-spot multi-party dialogue, providing comprehensive insights from both audience and performer experiences with AI on stage. Our human-in-the-loop methodology underlines the challenges of these LLMs in generating context-relevant responses, stressing the user interface's crucial role. Audience feedback indicates an evolving interest for AI-driven live entertainment, direct human-AI interaction, and a diverse range of expectations about AI's conversational competence and utility as a creativity support tool. Human performers express immense enthusiasm, varied satisfaction, and the evolving public opinion highlights mixed emotions about AI's role in arts.