SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras
This addresses the scalability issue in dynamic scene reconstruction for applications like immersive rendering, though it appears incremental as it builds on existing reconstruction methods.
The paper tackles the problem of high-quality 4D reconstruction from sparse camera setups, which typically require dense arrays of synchronized cameras, by proposing a framework that uses a Spatio-Temporal Distortion Field to model inconsistencies in generative observations. The result is a complete pipeline that achieves spatio-temporally consistent high-fidelity renderings and significantly outperforms existing approaches on multi-camera dynamic scene benchmarks.
High-quality 4D reconstruction enables photorealistic and immersive rendering of the dynamic real world. However, unlike static scenes that can be fully captured with a single camera, high-quality dynamic scenes typically require dense arrays of tens or even hundreds of synchronized cameras. Dependence on such costly lab setups severely limits practical scalability. The reliance on such costly lab setups severely limits practical scalability. To this end, we propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations. Our key innovation is the Spatio-Temporal Distortion Field, which provides a unified mechanism for modeling inconsistencies in generative observations across both spatial and temporal dimensions. Building on this, we develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs. We evaluate our method on multi-camera dynamic scene benchmarks, achieving spatio-temporally consistent high-fidelity renderings and significantly outperforming existing approaches.