MaskFlow: Discrete Flows For Flexible and Efficient Long Video Generation
This addresses the problem of efficient long video generation for AI and multimedia applications, representing an incremental improvement with novel method integration.
The paper tackles the challenge of generating long, high-quality videos by introducing MaskFlow, a framework that combines discrete representations with flow-matching, enabling efficient generation of videos ten times longer than training sequences while achieving competitive Frechet Video Distance scores on datasets like FaceForensics and Deepmind Lab.
Generating long, high-quality videos remains a challenge due to the complex interplay of spatial and temporal dynamics and hardware limitations. In this work, we introduce MaskFlow, a unified video generation framework that combines discrete representations with flow-matching to enable efficient generation of high-quality long videos. By leveraging a frame-level masking strategy during training, MaskFlow conditions on previously generated unmasked frames to generate videos with lengths ten times beyond that of the training sequences. MaskFlow does so very efficiently by enabling the use of fast Masked Generative Model (MGM)-style sampling and can be deployed in both fully autoregressive as well as full-sequence generation modes. We validate the quality of our method on the FaceForensics (FFS) and Deepmind Lab (DMLab) datasets and report Frechet Video Distance (FVD) competitive with state-of-the-art approaches. We also provide a detailed analysis on the sampling efficiency of our method and demonstrate that MaskFlow can be applied to both timestep-dependent and timestep-independent models in a training-free manner.