Lights, Camera, Consistency: A Multistage Pipeline for Character-Stable AI Video Stories
This addresses the problem of character inconsistency in AI-generated videos for applications like storytelling, though it is incremental as it builds on existing models with a novel pipeline.
The paper tackles the challenge of generating long, cohesive video stories with consistent characters in text-to-video AI by introducing a multistage pipeline that uses a production script and visual anchors, resulting in a character consistency score of 7.99 compared to 0.55 without anchoring.
Generating long, cohesive video stories with consistent characters is a significant challenge for current text-to-video AI. We introduce a method that approaches video generation in a filmmaker-like manner. Instead of creating a video in one step, our proposed pipeline first uses a large language model to generate a detailed production script. This script guides a text-to-image model in creating consistent visuals for each character, which then serve as anchors for a video generation model to synthesize each scene individually. Our baseline comparisons validate the necessity of this multi-stage decomposition; specifically, we observe that removing the visual anchoring mechanism results in a catastrophic drop in character consistency scores (from 7.99 to 0.55), confirming that visual priors are essential for identity preservation. Furthermore, we analyze cultural disparities in current models, revealing distinct biases in subject consistency and dynamic degree between Indian vs Western-themed generations.