DrawTalking: Building Interactive Worlds by Sketching and Speaking
This addresses the need for more accessible, code-free tools for creative exploration and authoring, though it appears incremental as an early prototype.
The authors tackled the problem of building interactive worlds without coding by introducing DrawTalking, which allows users to sketch and speak to create and control such environments, with an early study showing the mechanics resonate across various creative-exploratory use cases.
We introduce DrawTalking, an approach to building and controlling interactive worlds by sketching and speaking while telling stories. It emphasizes user control and flexibility, and gives programming-like capability without requiring code. An early open-ended study with our prototype shows that the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.