StorySync: Training-Free Subject Consistency in Text-to-Image Generation via Region Harmonization
This addresses the challenge of subject inconsistency in visual storytelling for users of text-to-image models, offering an efficient, training-free solution that avoids computational costs and model interference, though it is incremental as it builds on existing diffusion models.
The paper tackled the problem of maintaining subject consistency in text-to-image generation for visual storytelling, proposing a training-free method that uses masked cross-image attention sharing and regional feature harmonization, resulting in successful generation of visually consistent subjects across various scenarios while preserving model creativity.
Generating a coherent sequence of images that tells a visual story, using text-to-image diffusion models, often faces the critical challenge of maintaining subject consistency across all story scenes. Existing approaches, which typically rely on fine-tuning or retraining models, are computationally expensive, time-consuming, and often interfere with the model's pre-existing capabilities. In this paper, we follow a training-free approach and propose an efficient consistent-subject-generation method. This approach works seamlessly with pre-trained diffusion models by introducing masked cross-image attention sharing to dynamically align subject features across a batch of images, and Regional Feature Harmonization to refine visually similar details for improved subject consistency. Experimental results demonstrate that our approach successfully generates visually consistent subjects across a variety of scenarios while maintaining the creative abilities of the diffusion model.