GeoVideo: Introducing Geometric Regularization into Video Generation Model
This addresses the challenge of generating physically plausible 3D-consistent videos for applications in entertainment, simulation, and robotics, representing an incremental improvement over existing 2D-focused methods.
The paper tackles the problem of temporal inconsistency and structural artifacts in video generation by introducing geometric regularization losses based on per-frame depth prediction, resulting in significantly more stable and geometrically consistent videos across multiple datasets.
Recent advances in video generation have enabled the synthesis of high-quality and visually realistic clips using diffusion transformer models. However, most existing approaches operate purely in the 2D pixel space and lack explicit mechanisms for modeling 3D structures, often resulting in temporally inconsistent geometries, implausible motions, and structural artifacts. In this work, we introduce geometric regularization losses into video generation by augmenting latent diffusion models with per-frame depth prediction. We adopted depth as the geometric representation because of the great progress in depth prediction and its compatibility with image-based latent encoders. Specifically, to enforce structural consistency over time, we propose a multi-view geometric loss that aligns the predicted depth maps across frames within a shared 3D coordinate system. Our method bridges the gap between appearance generation and 3D structure modeling, leading to improved spatio-temporal coherence, shape consistency, and physical plausibility. Experiments across multiple datasets show that our approach produces significantly more stable and geometrically consistent results than existing baselines.