Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion
This work addresses the challenge of generating consistent long videos for applications in video synthesis, offering improvements over existing methods but is incremental as it builds on FIFO-Diffusion with novel components.
The paper tackles the problem of maintaining long-range temporal consistency in tuning-free long video generation by proposing Ouroboros-Diffusion, which introduces latent sampling, Subject-Aware Cross-Frame Attention, and self-recurrent guidance to enhance structural and subject consistency, achieving superior performance on the VBench benchmark in terms of subject consistency, motion smoothness, and temporal consistency.
The first-in-first-out (FIFO) video diffusion, built on a pre-trained text-to-video model, has recently emerged as an effective approach for tuning-free long video generation. This technique maintains a queue of video frames with progressively increasing noise, continuously producing clean frames at the queue's head while Gaussian noise is enqueued at the tail. However, FIFO-Diffusion often struggles to keep long-range temporal consistency in the generated videos due to the lack of correspondence modeling across frames. In this paper, we propose Ouroboros-Diffusion, a novel video denoising framework designed to enhance structural and content (subject) consistency, enabling the generation of consistent videos of arbitrary length. Specifically, we introduce a new latent sampling technique at the queue tail to improve structural consistency, ensuring perceptually smooth transitions among frames. To enhance subject consistency, we devise a Subject-Aware Cross-Frame Attention (SACFA) mechanism, which aligns subjects across frames within short segments to achieve better visual coherence. Furthermore, we introduce self-recurrent guidance. This technique leverages information from all previous cleaner frames at the front of the queue to guide the denoising of noisier frames at the end, fostering rich and contextual global information interaction. Extensive experiments of long video generation on the VBench benchmark demonstrate the superiority of our Ouroboros-Diffusion, particularly in terms of subject consistency, motion smoothness, and temporal consistency.