PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment
This addresses the limitation of practical applicability in robotics and media generation by enabling more realistic video synthesis, though it is incremental as it builds on existing video diffusion models.
The paper tackles the problem of generating temporally incoherent and physically unrealistic videos in Video Diffusion Models by proposing PhysAlign, which uses synthetic data and alignment techniques to achieve physics-coherent image-to-video generation, resulting in significant performance improvements over existing models on tasks requiring physical reasoning and temporal stability.
Video Diffusion Models (VDMs) offer a promising approach for simulating dynamic scenes and environments, with broad applications in robotics and media generation. However, existing models often generate temporally incoherent content that violates basic physical intuition, significantly limiting their practical applicability. We propose PhysAlign, an efficient framework for physics-coherent image-to-video (I2V) generation that explicitly addresses this limitation. To overcome the critical scarcity of physics-annotated videos, we first construct a fully controllable synthetic data generation pipeline based on rigid-body simulation, yielding a highly-curated dataset with accurate, fine-grained physics and 3D annotations. Leveraging this data, PhysAlign constructs a unified physical latent space by coupling explicit 3D geometry constraints with a Gram-based spatio-temporal relational alignment that extracts kinematic priors from video foundation models. Extensive experiments demonstrate that PhysAlign significantly outperforms existing VDMs on tasks requiring complex physical reasoning and temporal stability, without compromising zero-shot visual quality. PhysAlign shows the potential to bridge the gap between raw visual synthesis and rigid-body kinematics, establishing a practical paradigm for genuinely physics-grounded video generation. The project page is available at https://physalign.github.io/PhysAlign.