Unified Sensor Simulation for Autonomous Driving
This work addresses sensor simulation challenges for autonomous driving systems, providing a unified framework that improves geometric consistency and photorealism, though it is incremental as it builds on existing 3DGUT splatting methods.
The authors tackled the problem of sensor simulation for autonomous driving by introducing XSIM, a framework that extends 3DGUT splatting with rolling-shutter modeling and resolves issues like cyclic projection in spherical cameras, achieving state-of-the-art performance on datasets such as Waymo Open Dataset, Argoverse 2, and PandaSet.
In this work, we introduce \textbf{XSIM}, a sensor simulation framework for autonomous driving. XSIM extends 3DGUT splatting with a generalized rolling-shutter modeling tailored for autonomous driving applications. Our framework provides a unified and flexible formulation for appearance and geometric sensor modeling, enabling rendering of complex sensor distortions in dynamic environments. We identify spherical cameras, such as LiDARs, as a critical edge case for existing 3DGUT splatting due to cyclic projection and time discontinuities at azimuth boundaries leading to incorrect particle projection. To address this issue, we propose a phase modeling mechanism that explicitly accounts temporal and shape discontinuities of Gaussians projected by the Unscented Transform at azimuth borders. In addition, we introduce an extended 3D Gaussian representation that incorporates two distinct opacity parameters to resolve mismatches between geometry and color distributions. As a result, our framework provides enhanced scene representations with improved geometric consistency and photorealistic appearance. We evaluate our framework extensively on multiple autonomous driving datasets, including Waymo Open Dataset, Argoverse 2, and PandaSet. Our framework consistently outperforms strong recent baselines and achieves state-of-the-art performance across all datasets. The source code is publicly available at \href{https://github.com/whesense/XSIM}{https://github.com/whesense/XSIM}.