Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization
This addresses the challenge for writers using LLMs in creative worldbuilding by providing incremental visual tools to manage and control generated content.
The authors tackled the problem of writers being overwhelmed by many generated story elements and lacking precise control over LLM-generated worldbuilding content by introducing Patchview, a customizable visual system that uses magnets and dust metaphors to aid sensemaking and steering, showing through a user study that it supports exploration and alignment with user intentions.
Large language models (LLMs) can help writers build story worlds by generating world elements, such as factions, characters, and locations. However, making sense of many generated elements can be overwhelming. Moreover, if the user wants to precisely control aspects of generated elements that are difficult to specify verbally, prompting alone may be insufficient. We introduce Patchview, a customizable LLM-powered system that visually aids worldbuilding by allowing users to interact with story concepts and elements through the physical metaphor of magnets and dust. Elements in Patchview are visually dragged closer to concepts with high relevance, facilitating sensemaking. The user can also steer the generation with verbally elusive concepts by indicating the desired position of the element between concepts. When the user disagrees with the LLM's visualization and generation, they can correct those by repositioning the element. These corrections can be used to align the LLM's future behaviors to the user's perception. With a user study, we show that Patchview supports the sensemaking of world elements and steering of element generation, facilitating exploration during the worldbuilding process. Patchview provides insights on how customizable visual representation can help sensemake, steer, and align generative AI model behaviors with the user's intentions.