MTGS: Multi-Traversal Gaussian Splatting
This addresses the challenge of scene reconstruction for autonomous vehicle simulators using multi-traversal data, representing a strong specific gain in that domain.
The paper tackles the problem of reconstructing high-quality driving scenes from multi-traversal data, which often suffers from appearance variations and dynamic objects, by proposing MTGS, a method that models shared static geometry and handles dynamic elements separately, resulting in a 23.5% improvement in LPIPS and 46.3% improvement in geometry accuracy compared to baselines.
Multi-traversal data, commonly collected through daily commutes or by self-driving fleets, provides multiple viewpoints for scene reconstruction within a road block. This data offers significant potential for high-quality novel view synthesis, which is crucial for applications such as autonomous vehicle simulators. However, inherent challenges in multi-traversal data often result in suboptimal reconstruction quality, including variations in appearance and the presence of dynamic objects. To address these issues, we propose Multi-Traversal Gaussian Splatting (MTGS), a novel approach that reconstructs high-quality driving scenes from arbitrarily collected multi-traversal data by modeling a shared static geometry while separately handling dynamic elements and appearance variations. Our method employs a multi-traversal dynamic scene graph with a shared static node and traversal-specific dynamic nodes, complemented by color correction nodes with learnable spherical harmonics coefficient residuals. This approach enables high-fidelity novel view synthesis and provides flexibility to navigate any viewpoint. We conduct extensive experiments on a large-scale driving dataset, nuPlan, with multi-traversal data. Our results demonstrate that MTGS improves LPIPS by 23.5% and geometry accuracy by 46.3% compared to single-traversal baselines. The code and data would be available to the public.