Evaluating Design Video Generation: Metrics for Compositional Fidelity
It addresses the lack of standardized evaluation for design video generation, a domain-specific problem for researchers and practitioners in design animation.
The paper introduces an automated evaluation framework for design animation videos, measuring layout fidelity, motion correctness, temporal quality, and content fidelity, eliminating the need for subjective human evaluation.
Generative video models are increasingly used in design animation tasks, yet no standardized evaluation framework exists for this domain. Unlike natural video generation, design animation imposes structured constraints: specific components shall animate with prescribed motion types, directions, speed and timing, while non-animated regions must remain stable and layout structure must be preserved. This paper provides a fully automated evaluation framework organized across four dimensions: layout fidelity, motion correctness, temporal quality, and content fidelity. This eliminates the reliance on subjective human evaluation and establishes a common basis for benchmarking progress in the field.