PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR
This addresses the need for more interactive and playful AI co-authoring tools for writers, though it appears incremental as it builds on existing multimodal and AI-assisted systems.
The authors tackled the problem of AI writing tools poorly supporting spatial and interactive storytelling by developing PlayWrite, a mixed-reality system where users author stories through direct manipulation, resulting in a user study (N=13) showing it fosters improvisational and playful processes with AI as a collaborative partner.
Current AI writing tools, which rely on text prompts, poorly support the spatial and interactive nature of storytelling where ideas emerge from direct manipulation and play. We present PlayWrite, a mixed-reality system where users author stories by directly manipulating virtual characters and props. A multi-agent AI pipeline interprets these actions into Intent Frames -structured narrative beats visualized as rearrangeable story marbles on a timeline. A large language model then transforms the user's assembled sequence into a final narrative. A user study (N=13) with writers from varying domains found that PlayWrite fosters a highly improvisational and playful process. Users treated the AI as a collaborative partner, using its unexpected responses to spark new ideas and overcome creative blocks. PlayWrite demonstrates an approach for co-creative systems that move beyond text to embrace direct manipulation and play as core interaction modalities.