TaleForge: Interactive Multimodal System for Personalized Story Creation
This addresses the need for more immersive and user-centric storytelling experiences, though it is incremental in combining existing multimodal AI techniques.
The authors tackled the problem of passive and generic story creation by developing TaleForge, a system that integrates LLMs and diffusion models to embed users' facial images into personalized narratives and illustrations, resulting in heightened engagement and ownership as demonstrated in a user study.
Storytelling is a deeply personal and creative process, yet existing methods often treat users as passive consumers, offering generic plots with limited personalization. This undermines engagement and immersion, especially where individual style or appearance is crucial. We introduce TaleForge, a personalized story-generation system that integrates large language models (LLMs) and text-to-image diffusion to embed users' facial images within both narratives and illustrations. TaleForge features three interconnected modules: Story Generation, where LLMs create narratives and character descriptions from user prompts; Personalized Image Generation, merging users' faces and outfit choices into character illustrations; and Background Generation, creating scene backdrops that incorporate personalized characters. A user study demonstrated heightened engagement and ownership when individuals appeared as protagonists. Participants praised the system's real-time previews and intuitive controls, though they requested finer narrative editing tools. TaleForge advances multimodal storytelling by aligning personalized text and imagery to create immersive, user-centric experiences.