SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes
This addresses the challenge of modeling specular surfaces in dynamic 3D reconstruction for computer vision and graphics applications, representing an incremental improvement over existing 3DGS approaches.
The paper tackles the problem of accurately reconstructing dynamic specular scenes, which previous 3D Gaussian Splatting methods struggled with, by introducing a residual correction technique and deformable environment map, achieving photorealistic results that outperform state-of-the-art methods.
We present SpectroMotion, a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes. Previous methods extending 3DGS to model dynamic scenes have struggled to represent specular surfaces accurately. Our method addresses this limitation by introducing a residual correction technique for accurate surface normal computation during deformation, complemented by a deformable environment map that adapts to time-varying lighting conditions. We implement a coarse-to-fine training strategy significantly enhancing scene geometry and specular color prediction. It is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes, outperforming state-of-the-art methods in rendering complex, dynamic, and specular scenes.