A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction
This work addresses the problem of physical coherence in video generation models for researchers and developers in the field of computer vision and artificial intelligence.
The authors tackled the problem of evaluating video generation models' physical coherence, introducing a benchmark called PhyCoBench, and achieved a high alignment between automated and manual evaluations using their proposed PhyCoPredictor model. The benchmark includes 120 prompts covering 7 categories of physical principles and was tested on four state-of-the-art text-to-video models.
Recent advances in video generation models demonstrate their potential as world simulators, but they often struggle with videos deviating from physical laws, a key concern overlooked by most text-to-video benchmarks. We introduce a benchmark designed specifically to assess the Physical Coherence of generated videos, PhyCoBench. Our benchmark includes 120 prompts covering 7 categories of physical principles, capturing key physical laws observable in video content. We evaluated four state-of-the-art (SoTA) T2V models on PhyCoBench and conducted manual assessments. Additionally, we propose an automated evaluation model: PhyCoPredictor, a diffusion model that generates optical flow and video frames in a cascade manner. Through a consistency evaluation comparing automated and manual sorting, the experimental results show that PhyCoPredictor currently aligns most closely with human evaluation. Therefore, it can effectively evaluate the physical coherence of videos, providing insights for future model optimization. Our benchmark, including physical coherence prompts, the automatic evaluation tool PhyCoPredictor, and the generated video dataset, has been released on GitHub at https://github.com/Jeckinchen/PhyCoBench.