Open-Sora Plan: Open-Source Large Video Generation Model
This addresses the need for accessible, high-quality video generation tools for the research community, though it appears incremental as it builds on existing methods with new components.
The authors tackled the problem of generating high-resolution, long-duration videos from user inputs by developing Open-Sora Plan, an open-source large video generation model that achieves impressive results in qualitative and quantitative evaluations.
We introduce Open-Sora Plan, an open-source project that aims to contribute a large generation model for generating desired high-resolution videos with long durations based on various user inputs. Our project comprises multiple components for the entire video generation process, including a Wavelet-Flow Variational Autoencoder, a Joint Image-Video Skiparse Denoiser, and various condition controllers. Moreover, many assistant strategies for efficient training and inference are designed, and a multi-dimensional data curation pipeline is proposed for obtaining desired high-quality data. Benefiting from efficient thoughts, our Open-Sora Plan achieves impressive video generation results in both qualitative and quantitative evaluations. We hope our careful design and practical experience can inspire the video generation research community. All our codes and model weights are publicly available at \url{https://github.com/PKU-YuanGroup/Open-Sora-Plan}.