RoDyGS: Robust Dynamic Gaussian Splatting for Casual Videos
This work addresses the challenge of dynamic view synthesis for casual videos, which is incremental as it builds on existing Gaussian Splatting techniques.
The authors tackled the problem of optimizing dynamic neural fields from casual videos lacking 3D information, and their method, RoDyGS, significantly outperformed previous pose-free dynamic neural fields while achieving competitive rendering quality with pose-free static neural fields.
Dynamic view synthesis (DVS) has advanced remarkably in recent years, achieving high-fidelity rendering while reducing computational costs. Despite the progress, optimizing dynamic neural fields from casual videos remains challenging, as these videos do not provide direct 3D information, such as camera trajectories or the underlying scene geometry. In this work, we present RoDyGS, an optimization pipeline for dynamic Gaussian Splatting from casual videos. It effectively learns motion and underlying geometry of scenes by separating dynamic and static primitives, and ensures that the learned motion and geometry are physically plausible by incorporating motion and geometric regularization terms. We also introduce a comprehensive benchmark, Kubric-MRig, that provides extensive camera and object motion along with simultaneous multi-view captures, features that are absent in previous benchmarks. Experimental results demonstrate that the proposed method significantly outperforms previous pose-free dynamic neural fields and achieves competitive rendering quality compared to existing pose-free static neural fields. The code and data are publicly available at https://rodygs.github.io/.