MSG score: A Comprehensive Evaluation for Multi-Scene Video Generation
This addresses the need for comprehensive evaluation metrics in multi-scene video generation, which is incremental as it builds on existing video generation methods by automating manual selection processes.
The paper tackles the problem of evaluating multi-scene video generation by proposing an automated score-based benchmark to assess factors like character consistency and aesthetic quality, enabling more objective and efficient selection of high-quality videos.
This paper addresses the metrics required for generating multi-scene videos based on a continuous scenario, as opposed to traditional short video generation. Scenario-based videos require a comprehensive evaluation that considers multiple factors such as character consistency, artistic coherence, aesthetic quality, and the alignment of the generated content with the intended prompt. Additionally, in video generation, unlike single images, the movement of characters across frames introduces potential issues like distortion or unintended changes, which must be effectively evaluated and corrected. In the context of probabilistic models like diffusion, generating the desired scene requires repeated sampling and manual selection, akin to how a film director chooses the best shots from numerous takes. We propose a score-based evaluation benchmark that automates this process, enabling a more objective and efficient assessment of these complexities. This approach allows for the generation of high-quality multi-scene videos by selecting the best outcomes based on automated scoring rather than manual inspection.