Synthetic Video Enhances Physical Fidelity in Video Synthesis
This work addresses the challenge of generating physically realistic videos for applications in computer vision and graphics, though it is incremental as it builds on existing video synthesis methods.
The paper tackles the problem of enhancing physical fidelity in video generation models by leveraging synthetic videos from computer graphics pipelines, demonstrating efficacy in reducing artifacts and improving consistency across three tasks.
We investigate how to enhance the physical fidelity of video generation models by leveraging synthetic videos derived from computer graphics pipelines. These rendered videos respect real-world physics, such as maintaining 3D consistency, and serve as a valuable resource that can potentially improve video generation models. To harness this potential, we propose a solution that curates and integrates synthetic data while introducing a method to transfer its physical realism to the model, significantly reducing unwanted artifacts. Through experiments on three representative tasks emphasizing physical consistency, we demonstrate its efficacy in enhancing physical fidelity. While our model still lacks a deep understanding of physics, our work offers one of the first empirical demonstrations that synthetic video enhances physical fidelity in video synthesis. Website: https://kevinz8866.github.io/simulation/