MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving
This work addresses the need for high-fidelity simulation in autonomous driving to improve safety and performance, though it appears incremental as it builds on existing NeRF methods.
The authors tackled the problem of simulating realistic sensor data for autonomous driving to address corner cases by proposing a simulator based on neural radiance fields (NeRFs), which achieves state-of-the-art photo-realism and offers instance-aware, modular features.
Nowadays, autonomous cars can drive smoothly in ordinary cases, and it is widely recognized that realistic sensor simulation will play a critical role in solving remaining corner cases by simulating them. To this end, we propose an autonomous driving simulator based upon neural radiance fields (NeRFs). Compared with existing works, ours has three notable features: (1) Instance-aware. Our simulator models the foreground instances and background environments separately with independent networks so that the static (e.g., size and appearance) and dynamic (e.g., trajectory) properties of instances can be controlled separately. (2) Modular. Our simulator allows flexible switching between different modern NeRF-related backbones, sampling strategies, input modalities, etc. We expect this modular design to boost academic progress and industrial deployment of NeRF-based autonomous driving simulation. (3) Realistic. Our simulator set new state-of-the-art photo-realism results given the best module selection. Our simulator will be open-sourced while most of our counterparts are not. Project page: https://open-air-sun.github.io/mars/.