Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation
It provides a comprehensive overview for researchers in AI and video generation, highlighting gaps and future directions, but is incremental as it synthesizes existing work without new experimental results.
This survey systematically reviews 97 research articles on text-to-video generation, analyzing core building blocks and supporting features to assess progress toward AGI world models, and identifies key shortcomings in Sora-generated videos that call for more in-depth studies in areas like datasets and evaluation metrics.
The evolution of video generation from text, starting with animating MNIST numbers to simulating the physical world with Sora, has progressed at a breakneck speed over the past seven years. While often seen as a superficial expansion of the predecessor text-to-image generation model, text-to-video generation models are developed upon carefully engineered constituents. Here, we systematically discuss these elements consisting of but not limited to core building blocks (vision, language, and temporal) and supporting features from the perspective of their contributions to achieving a world model. We employ the PRISMA framework to curate 97 impactful research articles from renowned scientific databases primarily studying video synthesis using text conditions. Upon minute exploration of these manuscripts, we observe that text-to-video generation involves more intricate technologies beyond the plain extension of text-to-image generation. Our additional review into the shortcomings of Sora-generated videos pinpoints the call for more in-depth studies in various enabling aspects of video generation such as dataset, evaluation metric, efficient architecture, and human-controlled generation. Finally, we conclude that the study of the text-to-video generation may still be in its infancy, requiring contribution from the cross-discipline research community towards its advancement as the first step to realize artificial general intelligence (AGI).